[PATCH] S390: Refactor ifunc handling

[Stefan Liebler]

This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.

In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13". After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.

Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.

Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.

I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.

If no one objects, I plan to commit this series in the next one or two weeks.

[Stefan Liebler]

The renaming of hwcap arguments in ifunc-resolvers is needed
in order to prepare for further commits which refactors
ifunc handling for memset, memcmp, and memcpy. Now you are able
to use s390_libc_ifunc_init which stores the stfle bits
within the expression for an ifunc-resolver generated by
s390_libc_ifunc_expr.

ChangeLog:

* sysdeps/s390/multiarch/ifunc-resolve.h
(s390_libc_ifunc_init, s390_libc_ifunc,
s390_vx_libc_ifunc2_redirected): Use hwcap instead of dl_hwcap.
---
sysdeps/s390/multiarch/ifunc-resolve.h | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/sysdeps/s390/multiarch/ifunc-resolve.h b/sysdeps/s390/multiarch/ifunc-resolve.h
index b42ed922fd..6139bd39c5 100644
--- a/sysdeps/s390/multiarch/ifunc-resolve.h
+++ b/sysdeps/s390/multiarch/ifunc-resolve.h
@@ -42,9 +42,9 @@
: : "cc");
#define s390_libc_ifunc_init() \
unsigned long long stfle_bits = 0ULL; \
- if (__glibc_likely((dl_hwcap & HWCAP_S390_STFLE) \
- && (dl_hwcap & HWCAP_S390_ZARCH) \
- && (dl_hwcap & HWCAP_S390_HIGH_GPRS))) \
+ if (__glibc_likely((hwcap & HWCAP_S390_STFLE) \
+ && (hwcap & HWCAP_S390_ZARCH) \
+ && (hwcap & HWCAP_S390_HIGH_GPRS))) \
{ \
S390_STORE_STFLE (stfle_bits); \
}
@@ -61,7 +61,7 @@
: __glibc_likely (S390_IS_Z10 (stfle_bits)) \
? RESOLVERFUNC##_z10 \
: RESOLVERFUNC##_default, \
- unsigned long int dl_hwcap, s390_libc_ifunc_init);
+ unsigned long int hwcap, s390_libc_ifunc_init);

#define s390_vx_libc_ifunc(FUNC) \
s390_vx_libc_ifunc2_redirected(FUNC, FUNC, FUNC)
@@ -79,10 +79,10 @@
extern __typeof (TYPE_FUNC) RESOLVERFUNC##_vx attribute_hidden; \
extern __typeof (TYPE_FUNC) RESOLVERFUNC##_c attribute_hidden; \
__ifunc (TYPE_FUNC, FUNC, \
- (dl_hwcap & HWCAP_S390_VX) \
+ (hwcap & HWCAP_S390_VX) \
? RESOLVERFUNC##_vx \
: RESOLVERFUNC##_c, \
- unsigned long int dl_hwcap, s390_vx_libc_ifunc_init);
+ unsigned long int hwcap, s390_vx_libc_ifunc_init);

#define s390_libc_ifunc_expr_init()
#define s390_libc_ifunc_expr(TYPE_FUNC, FUNC, EXPR) \

[Florian Weimer]

Post by Stefan Liebler
+ if (__glibc_likely((hwcap & HWCAP_S390_STFLE) \

Missing space before paren. Since HWCAP_S390_STFLE 4 is four, this
__glibc_likely is actually equivalent to __glibc_unlikely, so perhaps
you should drop it.

Thanks,
Florian

[Stefan Liebler]

Post by Florian Weimer

Post by Stefan Liebler
+ if (__glibc_likely((hwcap & HWCAP_S390_STFLE) \

Missing space before paren. Since HWCAP_S390_STFLE 4 is four, this
__glibc_likely is actually equivalent to __glibc_unlikely, so perhaps
you should drop it.
Thanks,
Florian

Hi Florian,

I've added the missing space.

The condition of __glibc_likely does not result in the value "4"
as the full condition is:
(hwcap & HWCAP_S390_STFLE)
&& (hwcap & HWCAP_S390_ZARCH)
&& (hwcap & HWCAP_S390_HIGH_GPRS)

If hwcap is e.g. 518, then the result is one and __glibc_likely is not
equivalent to __glibc_unlikely.

Note: here are the other HWCAP_S390 defines:
#define HWCAP_S390_STFLE 4
#define HWCAP_S390_ZARCH 2
#define HWCAP_S390_HIGH_GPRS 512

Thanks.
Stefan

[Florian Weimer]

Post by Stefan Liebler
I've added the missing space.

Thanks.

Post by Stefan Liebler
The condition of __glibc_likely does not result in the value "4"
(hwcap & HWCAP_S390_STFLE)
&& (hwcap & HWCAP_S390_ZARCH)
&& (hwcap & HWCAP_S390_HIGH_GPRS)

You are right, of course. Sorry for the noise.

Florian

[Stefan Liebler]

Add a configure check for z10 in the same way as done for z196.

ChangeLog:

* config.h.in (HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT): New undefine.
* sysdeps/s390/configure.ac: Add check for z10 support.
* sysdeps/s390/configure: Regenerated.
---
config.h.in | 3 +++
sysdeps/s390/configure | 39 +++++++++++++++++++++++++++++++++++++++
sysdeps/s390/configure.ac | 29 +++++++++++++++++++++++++++++
3 files changed, 71 insertions(+)

diff --git a/config.h.in b/config.h.in
index 141db213a9..beecc39d5b 100644
--- a/config.h.in
+++ b/config.h.in
@@ -62,6 +62,9 @@
/* Define if assembler supports AVX512DQ. */
#undef HAVE_AVX512DQ_ASM_SUPPORT

+/* Define if assembler supports z10 zarch instructions as default on S390. */
+#undef HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
+
/* Define if assembler supports z196 zarch instructions as default on S390. */
#undef HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT

diff --git a/sysdeps/s390/configure b/sysdeps/s390/configure
index 74b415f2ab..f30f864436 100644
--- a/sysdeps/s390/configure
+++ b/sysdeps/s390/configure
@@ -112,6 +112,45 @@ then

fi

+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for S390 z10 zarch instruction support as default" >&5
+$as_echo_n "checking for S390 z10 zarch instruction support as default... " >&6; }
+if ${libc_cv_asm_s390_min_z10_zarch+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ cat > conftest.c <<\EOF
+void testinsn (void *a, void *b, int n)
+{
+ __asm__ ("exrl %2,1f \n\t"
+ "j 2f \n\t"
+ "1: mvc 0(1,%0),0(%1) \n\t"
+ "2:"
+ : : "a" (a), "a" (b), "d" (n)
+ : "memory", "cc");
+}
+EOF
+if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS --shared conftest.c
+ -o conftest.o &> /dev/null'
+ { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5
+ (eval $ac_try) 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; } ;
+then
+ libc_cv_asm_s390_min_z10_zarch=yes
+else
+ libc_cv_asm_s390_min_z10_zarch=no
+fi
+rm -f conftest*
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_asm_s390_min_z10_zarch" >&5
+$as_echo "$libc_cv_asm_s390_min_z10_zarch" >&6; }
+
+if test "$libc_cv_asm_s390_min_z10_zarch" = yes ;
+then
+ $as_echo "#define HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT 1" >>confdefs.h
+
+fi
+
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for S390 z196 zarch instruction support as default" >&5
$as_echo_n "checking for S390 z196 zarch instruction support as default... " >&6; }
if ${libc_cv_asm_s390_min_z196_zarch+:} false; then :
diff --git a/sysdeps/s390/configure.ac b/sysdeps/s390/configure.ac
index 1cdb021282..981f7a79dd 100644
--- a/sysdeps/s390/configure.ac
+++ b/sysdeps/s390/configure.ac
@@ -80,6 +80,35 @@ then
AC_DEFINE(HAVE_S390_VX_GCC_SUPPORT)
fi

+AC_CACHE_CHECK(for S390 z10 zarch instruction support as default,
+ libc_cv_asm_s390_min_z10_zarch, [dnl
+cat > conftest.c <<\EOF
+void testinsn (void *a, void *b, int n)
+{
+ __asm__ ("exrl %2,1f \n\t"
+ "j 2f \n\t"
+ "1: mvc 0(1,%0),0(%1) \n\t"
+ "2:"
+ : : "a" (a), "a" (b), "d" (n)
+ : "memory", "cc");
+}
+EOF
+dnl
+dnl test, if assembler supports S390 z10 zarch instructions as default
+if AC_TRY_COMMAND([${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS --shared conftest.c
+ -o conftest.o &> /dev/null]) ;
+then
+ libc_cv_asm_s390_min_z10_zarch=yes
+else
+ libc_cv_asm_s390_min_z10_zarch=no
+fi
+rm -f conftest* ])
+
+if test "$libc_cv_asm_s390_min_z10_zarch" = yes ;
+then
+ AC_DEFINE(HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT)
+fi
+
AC_CACHE_CHECK(for S390 z196 zarch instruction support as default,
libc_cv_asm_s390_min_z196_zarch, [dnl
cat > conftest.c <<\EOF

[Stefan Liebler]

The implementation of memset for s390-32 (31bit) and
s390-64 (64bit) is nearly the same.
This patch unifies it for maintability reasons.

__memset_z10 and __memset_z196 differs between 31 and 64bit:
-31bit needs .machinemode "zarch_nohighgprs" and llgfr %r4,%r4
-lr vs lgr and some other instructions:
But lgr and co can be also used on 31bit as this ifunc variant
is only called if we are on a zarch machine.

__memset_default differs between 31 and 64bit:
-Some 31bit vs 64bit instructions (e.g. ltr vs ltgr.
Solved with 31/64 specific instruction macros).
-The address of mvc instruction is setup in different ways
(larl vs bras). Solved with #if defined __s390x__.

Otherwise 31/64bit implementation has the same structure of the code.

ChangeLog:

* sysdeps/s390/s390-64/memset.S: Move to ...
* sysdeps/s390/memset.S: ... here.
Adjust to be usable for 31/64bit.
* sysdeps/s390/s390-32/memset.S: Delete File.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add memset.
* sysdeps/s390/s390-32/multiarch/Makefile (sysdep_routines):
Remove memset.
* sysdeps/s390/s390-64/multiarch/Makefile: Likewise.
* sysdeps/s390/s390-64/multiarch/memset-s390x.S: Move to ...
* sysdeps/s390/multiarch/memset-s390x.S: ... here.
Adjust to be usable for 31/64bit.
* sysdeps/s390/s390-32/multiarch/memset-s390.S: Delete File.
* sysdeps/s390/s390-64/multiarch/memset.c: Move to ...
* sysdeps/s390/multiarch/memset.c: ... here.
* sysdeps/s390/s390-32/multiarch/memset.c: Delete File.
---
sysdeps/s390/{s390-64 => }/memset.S | 67 +++++++---
sysdeps/s390/multiarch/Makefile | 3 +-
.../{s390-64 => }/multiarch/memset-s390x.S | 10 +-
sysdeps/s390/{s390-32 => }/multiarch/memset.c | 0
sysdeps/s390/s390-32/memset.S | 65 ----------
sysdeps/s390/s390-32/multiarch/Makefile | 3 +-
sysdeps/s390/s390-32/multiarch/memset-s390.S | 116 ------------------
sysdeps/s390/s390-64/multiarch/Makefile | 3 +-
sysdeps/s390/s390-64/multiarch/memset.c | 26 ----
9 files changed, 63 insertions(+), 230 deletions(-)
rename sysdeps/s390/{s390-64 => }/memset.S (58%)
rename sysdeps/s390/{s390-64 => }/multiarch/memset-s390x.S (90%)
rename sysdeps/s390/{s390-32 => }/multiarch/memset.c (100%)
delete mode 100644 sysdeps/s390/s390-32/memset.S
delete mode 100644 sysdeps/s390/s390-32/multiarch/memset-s390.S
delete mode 100644 sysdeps/s390/s390-64/multiarch/memset.c

diff --git a/sysdeps/s390/s390-64/memset.S b/sysdeps/s390/memset.S
similarity index 58%
rename from sysdeps/s390/s390-64/memset.S
rename to sysdeps/s390/memset.S
index 8799c6592c..72e7c5a42e 100644
--- a/sysdeps/s390/s390-64/memset.S
+++ b/sysdeps/s390/memset.S
@@ -1,4 +1,4 @@
-/* Set a block of memory to some byte value. 64 bit S/390 version.
+/* Set a block of memory to some byte value. 31/64 bit S/390 version.
Copyright (C) 2001-2018 Free Software Foundation, Inc.
Contributed by Martin Schwidefsky (***@de.ibm.com).
This file is part of the GNU C Library.
@@ -28,33 +28,60 @@

.text

+#if defined __s390x__
+# define LTGR ltgr
+# define CGHI cghi
+# define LGR lgr
+# define AGHI aghi
+# define BRCTG brctg
+#else
+# define LTGR ltr
+# define CGHI chi
+# define LGR lr
+# define AGHI ahi
+# define BRCTG brct
+#endif /* ! defined __s390x__ */
+
#ifdef USE_MULTIARCH
ENTRY(__memset_default)
#else
ENTRY(memset)
#endif
+#if defined __s390x__
.machine "z900"
- ltgr %r4,%r4
- je .L_Z900_4
+#else
+ .machine "g5"
+#endif /* ! defined __s390x__ */
+ LTGR %r4,%r4
+ je .L_Z900_G5_4
stc %r3,0(%r2)
- cghi %r4,1
- lgr %r1,%r2
- je .L_Z900_4
- aghi %r4,-2
+ CGHI %r4,1
+ LGR %r1,%r2
+ je .L_Z900_G5_4
+ AGHI %r4,-2
+#if defined __s390x__
+ larl %r5,.L_Z900_G5_18
srlg %r3,%r4,8
- ltgr %r3,%r3
- jne .L_Z900_14
-.L_Z900_3:
- larl %r3,.L_Z900_18
- ex %r4,0(%r3)
-.L_Z900_4:
+# define Z900_G5_EX_D 0
+#else
+ basr %r5,0
+.L_Z900_G5_19:
+# define Z900_G5_EX_D .L_Z900_G5_18-.L_Z900_G5_19
+ lr %r3,%r4
+ srl %r3,8
+#endif /* ! defined __s390x__ */
+ LTGR %r3,%r3
+ jne .L_Z900_G5_14
+.L_Z900_G5_3:
+ ex %r4,Z900_G5_EX_D(%r5)
+.L_Z900_G5_4:
br %r14
-.L_Z900_14:
+.L_Z900_G5_14:
mvc 1(256,%r1),0(%r1)
la %r1,256(%r1)
- brctg %r3,.L_Z900_14
- j .L_Z900_3
-.L_Z900_18:
+ BRCTG %r3,.L_Z900_G5_14
+ j .L_Z900_G5_3
+.L_Z900_G5_18:
mvc 1(1,%r1),0(%r1)
#ifdef USE_MULTIARCH
END(__memset_default)
@@ -62,3 +89,9 @@ END(__memset_default)
END(memset)
libc_hidden_builtin_def (memset)
#endif
+
+#undef LTGR
+#undef CGHI
+#undef LGR
+#undef AGHI
+#undef BRCTG
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index c893ebc565..93ad21bfa2 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -19,7 +19,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c \
- mempcpy
+ mempcpy \
+ memset memset-s390x
endif

ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/s390-64/multiarch/memset-s390x.S b/sysdeps/s390/multiarch/memset-s390x.S
similarity index 90%
rename from sysdeps/s390/s390-64/multiarch/memset-s390x.S
rename to sysdeps/s390/multiarch/memset-s390x.S
index 0c5aaef34f..aca3ac3fda 100644
--- a/sysdeps/s390/s390-64/multiarch/memset-s390x.S
+++ b/sysdeps/s390/multiarch/memset-s390x.S
@@ -1,4 +1,4 @@
-/* Set a block of memory to some byte value. 64 bit S/390 version.
+/* Set a block of memory to some byte value. 31/64 bit S/390 version.
Copyright (C) 2012-2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.

@@ -31,6 +31,10 @@

ENTRY(__memset_z196)
.machine "z196"
+ .machinemode "zarch_nohighgprs"
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
ltgr %r4,%r4
je .L_Z196_4
stc %r3,0(%r2)
@@ -61,6 +65,10 @@ END(__memset_z196)

ENTRY(__memset_z10)
.machine "z10"
+ .machinemode "zarch_nohighgprs"
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
cgije %r4,0,.L_Z10_4
stc %r3,0(%r2)
lgr %r1,%r2
diff --git a/sysdeps/s390/s390-32/multiarch/memset.c b/sysdeps/s390/multiarch/memset.c
similarity index 100%
rename from sysdeps/s390/s390-32/multiarch/memset.c
rename to sysdeps/s390/multiarch/memset.c
diff --git a/sysdeps/s390/s390-32/memset.S b/sysdeps/s390/s390-32/memset.S
deleted file mode 100644
index 57f08e1cae..0000000000
--- a/sysdeps/s390/s390-32/memset.S
+++ /dev/null
@@ -1,65 +0,0 @@
-/* Set a block of memory to some byte value. For IBM S390
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = address to memory area
- %r3 = byte to fill memory with
- %r4 = number of bytes to fill. */
-
- .text
-
-#ifdef USE_MULTIARCH
-ENTRY(__memset_default)
-#else
-ENTRY(memset)
-#endif
- .machine "g5"
- basr %r5,0
-.L_G5_19:
- ltr %r4,%r4
- je .L_G5_4
- stc %r3,0(%r2)
- chi %r4,1
- lr %r1,%r2
- je .L_G5_4
- ahi %r4,-2
- lr %r3,%r4
- srl %r3,8
- ltr %r3,%r3
- jne .L_G5_14
- ex %r4,.L_G5_20-.L_G5_19(%r5)
-.L_G5_4:
- br %r14
-.L_G5_14:
- mvc 1(256,%r1),0(%r1)
- la %r1,256(%r1)
- brct %r3,.L_G5_14
- ex %r4,.L_G5_20-.L_G5_19(%r5)
- j .L_G5_4
-.L_G5_20:
- mvc 1(1,%r1),0(%r1)
-#ifdef USE_MULTIARCH
-END(__memset_default)
-#else
-END(memset)
-libc_hidden_builtin_def (memset)
-#endif
diff --git a/sysdeps/s390/s390-32/multiarch/Makefile b/sysdeps/s390/s390-32/multiarch/Makefile
index f8aee14bbd..4b11e28656 100644
--- a/sysdeps/s390/s390-32/multiarch/Makefile
+++ b/sysdeps/s390/s390-32/multiarch/Makefile
@@ -1,4 +1,3 @@
ifeq ($(subdir),string)
-sysdep_routines += memset memset-s390 memcpy memcpy-s390 \
- memcmp memcmp-s390
+sysdep_routines += memcpy memcpy-s390 memcmp memcmp-s390
endif
diff --git a/sysdeps/s390/s390-32/multiarch/memset-s390.S b/sysdeps/s390/s390-32/multiarch/memset-s390.S
deleted file mode 100644
index b092073d6b..0000000000
--- a/sysdeps/s390/s390-32/multiarch/memset-s390.S
+++ /dev/null
@@ -1,116 +0,0 @@
-/* Set a block of memory to some byte value. 32 bit S/390 version.
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = address of memory area
- %r3 = byte to fill memory with
- %r4 = number of bytes to fill. */
-
- .text
-
-#if IS_IN (libc)
-
-ENTRY(__memset_z196)
- .machine "z196"
- .machinemode "zarch_nohighgprs"
- llgfr %r4,%r4
- ltgr %r4,%r4
- je .L_Z196_4
- stc %r3,0(%r2)
- lr %r1,%r2
- cghi %r4,1
- je .L_Z196_4
- aghi %r4,-2
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z196_1
-.L_Z196_3:
- exrl %r4,.L_Z196_17
-.L_Z196_4:
- br %r14
-.L_Z196_1:
- cgfi %r5,1048576
- jh __memset_mvcle # Switch to mvcle for >256MB
-.L_Z196_2:
- pfd 2,1024(%r1)
- mvc 1(256,%r1),0(%r1)
- aghi %r5,-1
- la %r1,256(%r1)
- jne .L_Z196_2
- j .L_Z196_3
-.L_Z196_17:
- mvc 1(1,%r1),0(%r1)
-END(__memset_z196)
-
-ENTRY(__memset_z10)
- .machine "z10"
- .machinemode "zarch_nohighgprs"
- llgfr %r4,%r4
- cgije %r4,0,.L_Z10_4
- stc %r3,0(%r2)
- lr %r1,%r2
- cgije %r4,1,.L_Z10_4
- aghi %r4,-2
- srlg %r5,%r4,8
- cgijlh %r5,0,.L_Z10_15
-.L_Z10_3:
- exrl %r4,.L_Z10_18
-.L_Z10_4:
- br %r14
-.L_Z10_15:
- cgfi %r5,163840 # Switch to mvcle for >40MB
- jh __memset_mvcle
-.L_Z10_14:
- pfd 2,1024(%r1)
- mvc 1(256,%r1),0(%r1)
- la %r1,256(%r1)
- brctg %r5,.L_Z10_14
- j .L_Z10_3
-.L_Z10_18:
- mvc 1(1,%r1),0(%r1)
-END(__memset_z10)
-
-ENTRY(__memset_mvcle)
- ahi %r4,2 # take back the change done by the caller
- lr %r0,%r2 # save source address
- lr %r1,%r3 # move pad byte to R1
- lr %r3,%r4
- sr %r4,%r4 # no source for MVCLE, only a pad byte
- sr %r5,%r5
-.L0: mvcle %r2,%r4,0(%r1) # thats it, MVCLE is your friend
- jo .L0
- lr %r2,%r0 # return value is source address
-.L1:
- br %r14
-END(__memset_mvcle)
-
-#endif /* IS_IN (libc) */
-
-#include "../memset.S"
-
-#if !IS_IN (libc)
-.globl memset
-.set memset,__memset_default
-#elif defined SHARED && IS_IN (libc)
-.globl __GI_memset
-.set __GI_memset,__memset_default
-#endif
diff --git a/sysdeps/s390/s390-64/multiarch/Makefile b/sysdeps/s390/s390-64/multiarch/Makefile
index 91053b5364..e4870c7ee1 100644
--- a/sysdeps/s390/s390-64/multiarch/Makefile
+++ b/sysdeps/s390/s390-64/multiarch/Makefile
@@ -1,4 +1,3 @@
ifeq ($(subdir),string)
-sysdep_routines += memset memset-s390x memcpy memcpy-s390x \
- memcmp memcmp-s390x
+sysdep_routines += memcpy memcpy-s390x memcmp memcmp-s390x
endif
diff --git a/sysdeps/s390/s390-64/multiarch/memset.c b/sysdeps/s390/s390-64/multiarch/memset.c
deleted file mode 100644
index 760b3e9df2..0000000000
--- a/sysdeps/s390/s390-64/multiarch/memset.c
+++ /dev/null
@@ -1,26 +0,0 @@
-/* Multiple versions of memset.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-#if IS_IN (libc)
-# define memset __redirect_memset
-# include <string.h>
-# undef memset
-# include <ifunc-resolve.h>
-
-s390_libc_ifunc (__redirect_memset, __memset, memset)
-#endif

[Stefan Liebler]

This patch removes the bzero s390 implementation with mvcle and
adds entry points for bzero in memset ifunc variants.
Therefore an ifunc resolver is implemented for bzero, too.

ChangeLog:

* sysdeps/s390/s390-32/bzero.S: Delete file.
* sysdeps/s390/s390-64/bzero.S: Likewise.
* sysdeps/s390/Makefile (sysdep_routines): Add bzero.
* sysdeps/s390/bzero.c: New file.
* sysdeps/s390/memset-z900.S: Add bzero entry points.
* sysdeps/s390/ifunc-memset.h: Add bzero function macros.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add bzero ifunc variants.
---
sysdeps/s390/Makefile | 2 +-
sysdeps/s390/{s390-32/bzero.S => bzero.c} | 53 +++++++++++++----------
sysdeps/s390/ifunc-memset.h | 9 ++++
sysdeps/s390/memset-z900.S | 37 +++++++++++++++-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 +++++++
sysdeps/s390/s390-64/bzero.S | 41 ------------------
6 files changed, 89 insertions(+), 68 deletions(-)
rename sysdeps/s390/{s390-32/bzero.S => bzero.c} (52%)
delete mode 100644 sysdeps/s390/s390-64/bzero.S

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index f90a41d3ae..31ae2ce418 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -54,5 +54,5 @@ endif
endif

ifeq ($(subdir),string)
-sysdep_routines += memset memset-z900
+sysdep_routines += bzero memset memset-z900
endif
diff --git a/sysdeps/s390/s390-32/bzero.S b/sysdeps/s390/bzero.c
similarity index 52%
rename from sysdeps/s390/s390-32/bzero.S
rename to sysdeps/s390/bzero.c
index 897aa2154a..9f8d95781b 100644
--- a/sysdeps/s390/s390-32/bzero.S
+++ b/sysdeps/s390/bzero.c
@@ -1,7 +1,6 @@
-/* bzero -- set a block of memory to zero. IBM S390 version
+/* Multiple versions of bzero.
+ Copyright (C) 2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.
- Copyright (C) 2000-2018 Free Software Foundation, Inc.
- Contributed by Martin Schwidefsky (***@de.ibm.com).

The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -17,26 +16,32 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-/*
- * R2 = address to memory area
- * R3 = number of bytes to fill
- */
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
- .text
-ENTRY(__bzero)
- ltr %r3,%r3
- jz .L1
- sr %r1,%r1 # set pad byte to zero
- sr %r4,%r4 # no source for MVCLE, only a pad byte
- sr %r5,%r5
-.L0: mvcle %r2,%r4,0(%r1) # thats it, MVCLE is your friend
- jo .L0
-.L1: br %r14
-END(__bzero)
-
-#ifndef NO_WEAK_ALIAS
+#include <ifunc-memset.h>
+#if HAVE_MEMSET_IFUNC
+# include <string.h>
+# include <ifunc-resolve.h>
+
+# if HAVE_MEMSET_Z900_G5
+extern __typeof (__bzero) BZERO_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_MEMSET_Z10
+extern __typeof (__bzero) BZERO_Z10 attribute_hidden;
+# endif
+
+# if HAVE_MEMSET_Z196
+extern __typeof (__bzero) BZERO_Z196 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__bzero, __bzero,
+ ({
+ s390_libc_ifunc_init ();
+ (HAVE_MEMSET_Z196 && S390_IS_Z196 (stfle_bits))
+ ? BZERO_Z196
+ : (HAVE_MEMSET_Z10 && S390_IS_Z10 (stfle_bits))
+ ? BZERO_Z10
+ : BZERO_DEFAULT;
+ })
+ )
weak_alias (__bzero, bzero)
#endif
diff --git a/sysdeps/s390/ifunc-memset.h b/sysdeps/s390/ifunc-memset.h
index 9a13b1001f..32bc155f7e 100644
--- a/sysdeps/s390/ifunc-memset.h
+++ b/sysdeps/s390/ifunc-memset.h
@@ -25,16 +25,19 @@

#if defined HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
# define MEMSET_DEFAULT MEMSET_Z196
+# define BZERO_DEFAULT BZERO_Z196
# define HAVE_MEMSET_Z900_G5 0
# define HAVE_MEMSET_Z10 0
# define HAVE_MEMSET_Z196 1
#elif defined HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
# define MEMSET_DEFAULT MEMSET_Z10
+# define BZERO_DEFAULT BZERO_Z10
# define HAVE_MEMSET_Z900_G5 0
# define HAVE_MEMSET_Z10 1
# define HAVE_MEMSET_Z196 HAVE_MEMSET_IFUNC
#else
# define MEMSET_DEFAULT MEMSET_Z900_G5
+# define BZERO_DEFAULT BZERO_Z900_G5
# define HAVE_MEMSET_Z900_G5 1
# define HAVE_MEMSET_Z10 HAVE_MEMSET_IFUNC
# define HAVE_MEMSET_Z196 HAVE_MEMSET_IFUNC
@@ -48,18 +51,24 @@

#if HAVE_MEMSET_Z900_G5
# define MEMSET_Z900_G5 __memset_default
+# define BZERO_Z900_G5 __bzero_default
#else
# define MEMSET_Z900_G5 NULL
+# define BZERO_Z900_G5 NULL
#endif

#if HAVE_MEMSET_Z10
# define MEMSET_Z10 __memset_z10
+# define BZERO_Z10 __bzero_z10
#else
# define MEMSET_Z10 NULL
+# define BZERO_Z10 NULL
#endif

#if HAVE_MEMSET_Z196
# define MEMSET_Z196 __memset_z196
+# define BZERO_Z196 __bzero_z196
#else
# define MEMSET_Z196 NULL
+# define BZERO_Z196 NULL
#endif
diff --git a/sysdeps/s390/memset-z900.S b/sysdeps/s390/memset-z900.S
index eaf13402bd..bfc659ae0b 100644
--- a/sysdeps/s390/memset-z900.S
+++ b/sysdeps/s390/memset-z900.S
@@ -22,10 +22,14 @@
#include "asm-syntax.h"
#include <ifunc-memset.h>

-/* INPUT PARAMETERS
+/* INPUT PARAMETERS - MEMSET
%r2 = address of memory area
%r3 = byte to fill memory with
- %r4 = number of bytes to fill. */
+ %r4 = number of bytes to fill.
+
+ INPUT PARAMETERS - BZERO
+ %r2 = address of memory area
+ %r3 = number of bytes to fill. */

.text

@@ -44,7 +48,14 @@
# define BRCTG brct
# endif /* ! defined __s390x__ */

+ENTRY(BZERO_Z900_G5)
+ LGR %r4,%r3
+ xr %r3,%r3
+ j .L_Z900_G5_start
+END(BZERO_Z900_G5)
+
ENTRY(MEMSET_Z900_G5)
+.L_Z900_G5_start:
#if defined __s390x__
.machine "z900"
#else
@@ -90,7 +101,16 @@ END(MEMSET_Z900_G5)
#endif /* HAVE_MEMSET_Z900_G5 */

#if HAVE_MEMSET_Z10
+ENTRY(BZERO_Z10)
+ .machine "z10"
+ .machinemode "zarch_nohighgprs"
+ lgr %r4,%r3
+ xr %r3,%r3
+ j .L_Z10_start
+END(BZERO_Z10)
+
ENTRY(MEMSET_Z10)
+.L_Z10_start:
.machine "z10"
.machinemode "zarch_nohighgprs"
# if !defined __s390x__
@@ -122,7 +142,16 @@ END(MEMSET_Z10)
#endif /* HAVE_MEMSET_Z10 */

#if HAVE_MEMSET_Z196
+ENTRY(BZERO_Z196)
+ .machine "z196"
+ .machinemode "zarch_nohighgprs"
+ lgr %r4,%r3
+ xr %r3,%r3
+ j .L_Z196_start
+END(BZERO_Z196)
+
ENTRY(MEMSET_Z196)
+.L_Z196_start:
.machine "z196"
.machinemode "zarch_nohighgprs"
# if !defined __s390x__
@@ -177,6 +206,10 @@ END(__memset_mvcle)
/* If we don't use ifunc, define an alias for memset here.
Otherwise see sysdeps/s390/memset.c. */
strong_alias (MEMSET_DEFAULT, memset)
+/* Same for bzero. If ifunc is used, see
+ sysdeps/s390/bzero.c. */
+strong_alias (BZERO_DEFAULT, __bzero)
+weak_alias (__bzero, bzero)
#endif

#if defined SHARED && IS_IN (libc)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 2f671eac1f..253f36045b 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -59,6 +59,21 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
# endif
# if HAVE_MEMSET_Z900_G5
IFUNC_IMPL_ADD (array, i, memset, 1, MEMSET_Z900_G5)
+# endif
+ )
+
+ /* Note: bzero is implemented in memset. */
+ IFUNC_IMPL (i, name, bzero,
+# if HAVE_MEMSET_Z196
+ IFUNC_IMPL_ADD (array, i, bzero,
+ S390_IS_Z196 (stfle_bits), BZERO_Z196)
+# endif
+# if HAVE_MEMSET_Z10
+ IFUNC_IMPL_ADD (array, i, bzero,
+ S390_IS_Z10 (stfle_bits), BZERO_Z10)
+# endif
+# if HAVE_MEMSET_Z900_G5
+ IFUNC_IMPL_ADD (array, i, bzero, 1, BZERO_Z900_G5)
# endif
)
#endif /* HAVE_MEMSET_IFUNC */
diff --git a/sysdeps/s390/s390-64/bzero.S b/sysdeps/s390/s390-64/bzero.S
deleted file mode 100644
index b321665298..0000000000
--- a/sysdeps/s390/s390-64/bzero.S
+++ /dev/null
@@ -1,41 +0,0 @@
-/* bzero -- set a block of memory to zero. 64 bit S/390 version.
- This file is part of the GNU C Library.
- Copyright (C) 2001-2018 Free Software Foundation, Inc.
- Contributed by Martin Schwidefsky (***@de.ibm.com).
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* INPUT PARAMETERS
- %r2 = address of memory area
- %r3 = number of bytes to fill. */
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
- .text
-ENTRY(__bzero)
- ltgr %r3,%r3
- jz .L1
- sgr %r1,%r1 # set pad byte to zero
- sgr %r4,%r4 # no source for MVCLE, only a pad byte
- sgr %r5,%r5
-.L0: mvcle %r2,%r4,0(%r1) # thats it, MVCLE is your friend
- jo .L0
-.L1: br %r14
-END(__bzero)
-
-#ifndef NO_WEAK_ALIAS
-weak_alias (__bzero, bzero)
-#endif

[Stefan Liebler]

This patch moves all ifunc variants for memset
to sysdeps/s390/memset-z900.S. The configure-check/preprocessor logic
in sysdeps/s390/ifunc-memset.h decides if ifunc is needed at all
and which ifunc variants should be available.
E.g. if the compiler/assembler already supports z196 by default,
the older ifunc variants are not included.
If we only need the newest ifunc variant,
then we can skip ifunc at all.

Therefore the ifunc-resolvers and __libc_ifunc_impl_list are adjusted
in order to handle only the available ifunc variants.

ChangeLog:

* sysdeps/s390/ifunc-memset.h: New File.
* sysdeps/s390/memset.S: Move to ...
* sysdeps/s390/memset-z900.S ... here.
Move implementations from memset-s390x.s to here.
* sysdeps/s390/multiarch/memset-s390x.S: Delete File.
* sysdeps/s390/multiarch/Makefile (sysdep_routines):
Remove memset variants.
* sysdeps/s390/Makefile (sysdep_routines):
Add memset variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Adjust ifunc variants for
memset.
* sysdeps/s390/multiarch/memset.c: Move ifunc resolver
to ...
* sysdeps/s390/memset.c: ... here.
Adjust ifunc variants for memset.
---
sysdeps/s390/Makefile | 4 +
sysdeps/s390/ifunc-memset.h | 65 ++++++++
.../memset-s390x.S => memset-z900.S} | 150 +++++++++++++-----
sysdeps/s390/memset.S | 97 -----------
sysdeps/s390/{multiarch => }/memset.c | 26 ++-
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 16 +-
7 files changed, 215 insertions(+), 146 deletions(-)
create mode 100644 sysdeps/s390/ifunc-memset.h
rename sysdeps/s390/{multiarch/memset-s390x.S => memset-z900.S} (57%)
delete mode 100644 sysdeps/s390/memset.S
rename sysdeps/s390/{multiarch => }/memset.c (60%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 496ab51973..f90a41d3ae 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -52,3 +52,7 @@ tst-dl-runtime-profile-noaudit-ENV = $(env-profile)
tst-dl-runtime-profile-audit-ENV = $(env-profile) $(env-audit)
endif
endif
+
+ifeq ($(subdir),string)
+sysdep_routines += memset memset-z900
+endif
diff --git a/sysdeps/s390/ifunc-memset.h b/sysdeps/s390/ifunc-memset.h
new file mode 100644
index 0000000000..9a13b1001f
--- /dev/null
+++ b/sysdeps/s390/ifunc-memset.h
@@ -0,0 +1,65 @@
+/* memset variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
+# define HAVE_MEMSET_IFUNC 1
+#else
+# define HAVE_MEMSET_IFUNC 0
+#endif
+
+#if defined HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
+# define MEMSET_DEFAULT MEMSET_Z196
+# define HAVE_MEMSET_Z900_G5 0
+# define HAVE_MEMSET_Z10 0
+# define HAVE_MEMSET_Z196 1
+#elif defined HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
+# define MEMSET_DEFAULT MEMSET_Z10
+# define HAVE_MEMSET_Z900_G5 0
+# define HAVE_MEMSET_Z10 1
+# define HAVE_MEMSET_Z196 HAVE_MEMSET_IFUNC
+#else
+# define MEMSET_DEFAULT MEMSET_Z900_G5
+# define HAVE_MEMSET_Z900_G5 1
+# define HAVE_MEMSET_Z10 HAVE_MEMSET_IFUNC
+# define HAVE_MEMSET_Z196 HAVE_MEMSET_IFUNC
+#endif
+
+#if HAVE_MEMSET_Z10 || HAVE_MEMSET_Z196
+# define HAVE_MEMSET_MVCLE 1
+#else
+# define HAVE_MEMSET_MVCLE 0
+#endif
+
+#if HAVE_MEMSET_Z900_G5
+# define MEMSET_Z900_G5 __memset_default
+#else
+# define MEMSET_Z900_G5 NULL
+#endif
+
+#if HAVE_MEMSET_Z10
+# define MEMSET_Z10 __memset_z10
+#else
+# define MEMSET_Z10 NULL
+#endif
+
+#if HAVE_MEMSET_Z196
+# define MEMSET_Z196 __memset_z196
+#else
+# define MEMSET_Z196 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/memset-s390x.S b/sysdeps/s390/memset-z900.S
similarity index 57%
rename from sysdeps/s390/multiarch/memset-s390x.S
rename to sysdeps/s390/memset-z900.S
index aca3ac3fda..eaf13402bd 100644
--- a/sysdeps/s390/multiarch/memset-s390x.S
+++ b/sysdeps/s390/memset-z900.S
@@ -1,5 +1,6 @@
/* Set a block of memory to some byte value. 31/64 bit S/390 version.
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
+ Copyright (C) 2001-2018 Free Software Foundation, Inc.
+ Contributed by Martin Schwidefsky (***@de.ibm.com).
This file is part of the GNU C Library.

The GNU C Library is free software; you can redistribute it and/or
@@ -17,8 +18,9 @@
<http://www.gnu.org/licenses/>. */


-#include "sysdep.h"
+#include <sysdep.h>
#include "asm-syntax.h"
+#include <ifunc-memset.h>

/* INPUT PARAMETERS
%r2 = address of memory area
@@ -27,43 +29,68 @@

.text

-#if IS_IN (libc)
+#if HAVE_MEMSET_Z900_G5
+# if defined __s390x__
+# define LTGR ltgr
+# define CGHI cghi
+# define LGR lgr
+# define AGHI aghi
+# define BRCTG brctg
+# else
+# define LTGR ltr
+# define CGHI chi
+# define LGR lr
+# define AGHI ahi
+# define BRCTG brct
+# endif /* ! defined __s390x__ */

-ENTRY(__memset_z196)
- .machine "z196"
- .machinemode "zarch_nohighgprs"
-# if !defined __s390x__
- llgfr %r4,%r4
-# endif /* !defined __s390x__ */
- ltgr %r4,%r4
- je .L_Z196_4
+ENTRY(MEMSET_Z900_G5)
+#if defined __s390x__
+ .machine "z900"
+#else
+ .machine "g5"
+#endif /* ! defined __s390x__ */
+ LTGR %r4,%r4
+ je .L_Z900_G5_4
stc %r3,0(%r2)
- lgr %r1,%r2
- cghi %r4,1
- je .L_Z196_4
- aghi %r4,-2
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z196_1
-.L_Z196_3:
- exrl %r4,.L_Z196_17
-.L_Z196_4:
+ CGHI %r4,1
+ LGR %r1,%r2
+ je .L_Z900_G5_4
+ AGHI %r4,-2
+#if defined __s390x__
+ larl %r5,.L_Z900_G5_18
+ srlg %r3,%r4,8
+# define Z900_G5_EX_D 0
+#else
+ basr %r5,0
+.L_Z900_G5_19:
+# define Z900_G5_EX_D .L_Z900_G5_18-.L_Z900_G5_19
+ lr %r3,%r4
+ srl %r3,8
+#endif /* ! defined __s390x__ */
+ LTGR %r3,%r3
+ jne .L_Z900_G5_14
+.L_Z900_G5_3:
+ ex %r4,Z900_G5_EX_D(%r5)
+.L_Z900_G5_4:
br %r14
-.L_Z196_1:
- cgfi %r5,1048576
- jh __memset_mvcle # Switch to mvcle for >256MB
-.L_Z196_2:
- pfd 2,1024(%r1)
+.L_Z900_G5_14:
mvc 1(256,%r1),0(%r1)
- aghi %r5,-1
la %r1,256(%r1)
- jne .L_Z196_2
- j .L_Z196_3
-.L_Z196_17:
+ BRCTG %r3,.L_Z900_G5_14
+ j .L_Z900_G5_3
+.L_Z900_G5_18:
mvc 1(1,%r1),0(%r1)
-END(__memset_z196)
+END(MEMSET_Z900_G5)
+# undef LTGR
+# undef CGHI
+# undef LGR
+# undef AGHI
+# undef BRCTG
+#endif /* HAVE_MEMSET_Z900_G5 */

-ENTRY(__memset_z10)
+#if HAVE_MEMSET_Z10
+ENTRY(MEMSET_Z10)
.machine "z10"
.machinemode "zarch_nohighgprs"
# if !defined __s390x__
@@ -91,8 +118,46 @@ ENTRY(__memset_z10)
j .L_Z10_3
.L_Z10_18:
mvc 1(1,%r1),0(%r1)
-END(__memset_z10)
+END(MEMSET_Z10)
+#endif /* HAVE_MEMSET_Z10 */
+
+#if HAVE_MEMSET_Z196
+ENTRY(MEMSET_Z196)
+ .machine "z196"
+ .machinemode "zarch_nohighgprs"
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+ ltgr %r4,%r4
+ je .L_Z196_4
+ stc %r3,0(%r2)
+ lgr %r1,%r2
+ cghi %r4,1
+ je .L_Z196_4
+ aghi %r4,-2
+ srlg %r5,%r4,8
+ ltgr %r5,%r5
+ jne .L_Z196_1
+.L_Z196_3:
+ exrl %r4,.L_Z196_17
+.L_Z196_4:
+ br %r14
+.L_Z196_1:
+ cgfi %r5,1048576
+ jh __memset_mvcle # Switch to mvcle for >256MB
+.L_Z196_2:
+ pfd 2,1024(%r1)
+ mvc 1(256,%r1),0(%r1)
+ aghi %r5,-1
+ la %r1,256(%r1)
+ jne .L_Z196_2
+ j .L_Z196_3
+.L_Z196_17:
+ mvc 1(1,%r1),0(%r1)
+END(MEMSET_Z196)
+#endif /* HAVE_MEMSET_Z196 */

+#if HAVE_MEMSET_MVCLE
ENTRY(__memset_mvcle)
aghi %r4,2 # take back the change done by the caller
lgr %r0,%r2 # save source address
@@ -106,15 +171,16 @@ ENTRY(__memset_mvcle)
.L1:
br %r14
END(__memset_mvcle)
+#endif /* HAVE_MEMSET_MVCLE */

-#endif /* IS_IN (libc) */
-
-#include "../memset.S"
+#if ! HAVE_MEMSET_IFUNC
+/* If we don't use ifunc, define an alias for memset here.
+ Otherwise see sysdeps/s390/memset.c. */
+strong_alias (MEMSET_DEFAULT, memset)
+#endif

-#if !IS_IN (libc)
-.globl memset
-.set memset,__memset_default
-#elif defined SHARED && IS_IN (libc)
-.globl __GI_memset
-.set __GI_memset,__memset_default
+#if defined SHARED && IS_IN (libc)
+/* Defines the internal symbol.
+ Compare to libc_hidden_builtin_def (memset) in string/memset.c. */
+strong_alias (MEMSET_DEFAULT, __GI_memset)
#endif
diff --git a/sysdeps/s390/memset.S b/sysdeps/s390/memset.S
deleted file mode 100644
index 72e7c5a42e..0000000000
--- a/sysdeps/s390/memset.S
+++ /dev/null
@@ -1,97 +0,0 @@
-/* Set a block of memory to some byte value. 31/64 bit S/390 version.
- Copyright (C) 2001-2018 Free Software Foundation, Inc.
- Contributed by Martin Schwidefsky (***@de.ibm.com).
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include <sysdep.h>
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = address of memory area
- %r3 = byte to fill memory with
- %r4 = number of bytes to fill. */
-
- .text
-
-#if defined __s390x__
-# define LTGR ltgr
-# define CGHI cghi
-# define LGR lgr
-# define AGHI aghi
-# define BRCTG brctg
-#else
-# define LTGR ltr
-# define CGHI chi
-# define LGR lr
-# define AGHI ahi
-# define BRCTG brct
-#endif /* ! defined __s390x__ */
-
-#ifdef USE_MULTIARCH
-ENTRY(__memset_default)
-#else
-ENTRY(memset)
-#endif
-#if defined __s390x__
- .machine "z900"
-#else
- .machine "g5"
-#endif /* ! defined __s390x__ */
- LTGR %r4,%r4
- je .L_Z900_G5_4
- stc %r3,0(%r2)
- CGHI %r4,1
- LGR %r1,%r2
- je .L_Z900_G5_4
- AGHI %r4,-2
-#if defined __s390x__
- larl %r5,.L_Z900_G5_18
- srlg %r3,%r4,8
-# define Z900_G5_EX_D 0
-#else
- basr %r5,0
-.L_Z900_G5_19:
-# define Z900_G5_EX_D .L_Z900_G5_18-.L_Z900_G5_19
- lr %r3,%r4
- srl %r3,8
-#endif /* ! defined __s390x__ */
- LTGR %r3,%r3
- jne .L_Z900_G5_14
-.L_Z900_G5_3:
- ex %r4,Z900_G5_EX_D(%r5)
-.L_Z900_G5_4:
- br %r14
-.L_Z900_G5_14:
- mvc 1(256,%r1),0(%r1)
- la %r1,256(%r1)
- BRCTG %r3,.L_Z900_G5_14
- j .L_Z900_G5_3
-.L_Z900_G5_18:
- mvc 1(1,%r1),0(%r1)
-#ifdef USE_MULTIARCH
-END(__memset_default)
-#else
-END(memset)
-libc_hidden_builtin_def (memset)
-#endif
-
-#undef LTGR
-#undef CGHI
-#undef LGR
-#undef AGHI
-#undef BRCTG
diff --git a/sysdeps/s390/multiarch/memset.c b/sysdeps/s390/memset.c
similarity index 60%
rename from sysdeps/s390/multiarch/memset.c
rename to sysdeps/s390/memset.c
index 760b3e9df2..57a35aebc7 100644
--- a/sysdeps/s390/multiarch/memset.c
+++ b/sysdeps/s390/memset.c
@@ -16,11 +16,33 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if IS_IN (libc)
+#include <ifunc-memset.h>
+#if HAVE_MEMSET_IFUNC
# define memset __redirect_memset
# include <string.h>
# undef memset
# include <ifunc-resolve.h>

-s390_libc_ifunc (__redirect_memset, __memset, memset)
+# if HAVE_MEMSET_Z900_G5
+extern __typeof (__redirect_memset) MEMSET_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_MEMSET_Z10
+extern __typeof (__redirect_memset) MEMSET_Z10 attribute_hidden;
+# endif
+
+# if HAVE_MEMSET_Z196
+extern __typeof (__redirect_memset) MEMSET_Z196 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect_memset, memset,
+ ({
+ s390_libc_ifunc_init ();
+ (HAVE_MEMSET_Z196 && S390_IS_Z196 (stfle_bits))
+ ? MEMSET_Z196
+ : (HAVE_MEMSET_Z10 && S390_IS_Z10 (stfle_bits))
+ ? MEMSET_Z10
+ : MEMSET_DEFAULT;
+ })
+ )
#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 93ad21bfa2..c893ebc565 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -19,8 +19,7 @@ sysdep_routines += strlen strlen-vx strlen-c \
rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c \
- mempcpy \
- memset memset-s390x
+ mempcpy
endif

ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index ec3373ae26..2f671eac1f 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -21,6 +21,7 @@
#include <wchar.h>
#include <ifunc-impl-list.h>
#include <ifunc-resolve.h>
+#include <ifunc-memset.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -46,12 +47,21 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
S390_STORE_STFLE (stfle_bits);
}

+#if HAVE_MEMSET_IFUNC
IFUNC_IMPL (i, name, memset,
+# if HAVE_MEMSET_Z196
IFUNC_IMPL_ADD (array, i, memset,
- S390_IS_Z196 (stfle_bits), __memset_z196)
+ S390_IS_Z196 (stfle_bits), MEMSET_Z196)
+# endif
+# if HAVE_MEMSET_Z10
IFUNC_IMPL_ADD (array, i, memset,
- S390_IS_Z10 (stfle_bits), __memset_z10)
- IFUNC_IMPL_ADD (array, i, memset, 1, __memset_default))
+ S390_IS_Z10 (stfle_bits), MEMSET_Z10)
+# endif
+# if HAVE_MEMSET_Z900_G5
+ IFUNC_IMPL_ADD (array, i, memset, 1, MEMSET_Z900_G5)
+# endif
+ )
+#endif /* HAVE_MEMSET_IFUNC */

IFUNC_IMPL (i, name, memcmp,
IFUNC_IMPL_ADD (array, i, memcmp,

[Stefan Liebler]

The implementation of memcmp for s390-32 (31bit) and
s390-64 (64bit) is nearly the same.
This patch unifies it for maintability reasons.

__memcmp_z10 and __memcmp_z196 differs between 31 and 64bit:
-31bit needs .machinemode "zarch_nohighgprs" and llgfr %r4,%r4
-lr vs lgr and some other instructions:
But lgr and co can be also used on 31bit as this ifunc variant
is only called if we are on a zarch machine.

__memcmp_default differs between 31 and 64bit:
-Some 31bit vs 64bit instructions (e.g. ltr vs ltgr.
Solved with 31/64 specific instruction macros).
-The address of mvc instruction is setup in different ways
(larl vs bras). Solved with #if defined __s390x__.

Otherwise 31/64bit implementation has the same structure of the code.

ChangeLog:

* sysdeps/s390/s390-64/memcmp.S: Move to ...
* sysdeps/s390/memcmp.S: ... here.
Adjust to be usable for 31/64bit.
* sysdeps/s390/s390-32/memcmp.S: Delete File.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add memcmp.
* sysdeps/s390/s390-32/multiarch/Makefile (sysdep_routines):
Remove memcmp.
* sysdeps/s390/s390-64/multiarch/Makefile: Likewise.
* sysdeps/s390/s390-64/multiarch/memcmp-s390x.S: Move to ...
* sysdeps/s390/multiarch/memcmp-s390x.S: ... here.
Adjust to be usable for 31/64bit.
* sysdeps/s390/s390-32/multiarch/memcmp-s390.S: Delete File.
* sysdeps/s390/s390-64/multiarch/memcmp.c: Move to ...
* sysdeps/s390/multiarch/memcmp.c: ... here.
* sysdeps/s390/s390-32/multiarch/memcmp.c: Delete File.
---
sysdeps/s390/{s390-64 => }/memcmp.S | 62 +++++++---
sysdeps/s390/multiarch/Makefile | 3 +-
.../{s390-64 => }/multiarch/memcmp-s390x.S | 10 +-
sysdeps/s390/{s390-32 => }/multiarch/memcmp.c | 0
sysdeps/s390/s390-32/memcmp.S | 66 -----------
sysdeps/s390/s390-32/multiarch/Makefile | 2 +-
sysdeps/s390/s390-32/multiarch/memcmp-s390.S | 107 ------------------
sysdeps/s390/s390-64/multiarch/Makefile | 2 +-
sysdeps/s390/s390-64/multiarch/memcmp.c | 27 -----
9 files changed, 60 insertions(+), 219 deletions(-)
rename sysdeps/s390/{s390-64 => }/memcmp.S (60%)
rename sysdeps/s390/{s390-64 => }/multiarch/memcmp-s390x.S (89%)
rename sysdeps/s390/{s390-32 => }/multiarch/memcmp.c (100%)
delete mode 100644 sysdeps/s390/s390-32/memcmp.S
delete mode 100644 sysdeps/s390/s390-32/multiarch/memcmp-s390.S
delete mode 100644 sysdeps/s390/s390-64/multiarch/memcmp.c

diff --git a/sysdeps/s390/s390-64/memcmp.S b/sysdeps/s390/memcmp.S
similarity index 60%
rename from sysdeps/s390/s390-64/memcmp.S
rename to sysdeps/s390/memcmp.S
index 005b19de45..751293a99e 100644
--- a/sysdeps/s390/s390-64/memcmp.S
+++ b/sysdeps/s390/memcmp.S
@@ -1,4 +1,4 @@
-/* memcmp - compare two memory blocks. 64 bit S/390 version.
+/* memcmp - compare two memory blocks. 31/64 bit S/390 version.
Copyright (C) 2012-2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.

@@ -26,34 +26,62 @@
%r4 = number of bytes to compare. */

.text
+#if defined __s390x__
+# define LTGR ltgr
+# define AGHI aghi
+# define BRCTG brctg
+#else
+# define LTGR ltr
+# define AGHI ahi
+# define BRCTG brct
+#endif /* ! defined __s390x__ */
+
#ifdef USE_MULTIARCH
ENTRY(__memcmp_default)
#else
ENTRY(memcmp)
#endif
+#if defined __s390x__
.machine "z900"
- ltgr %r4,%r4
- je .L_Z900_4
- aghi %r4,-1
+#else
+ .machine "g5"
+ basr %r5,0
+.L_Z900_G5_16:
+# define Z900_G5_EX_D .L_Z900_G5_15-.L_Z900_G5_16
+#endif /* ! defined __s390x__ */
+ LTGR %r4,%r4
+ je .L_Z900_G5_4
+ AGHI %r4,-1
+#if defined __s390x__
srlg %r1,%r4,8
- ltgr %r1,%r1
- jne .L_Z900_12
-.L_Z900_3:
- larl %r1,.L_Z900_15
- ex %r4,0(%r1)
-.L_Z900_4:
+ larl %r5,.L_Z900_G5_15
+# define Z900_G5_EX_D 0
+#else
+ lr %r1,%r4
+ srl %r1,8
+#endif /* ! defined __s390x__ */
+ LTGR %r1,%r1
+ jne .L_Z900_G5_12
+.L_Z900_G5_3:
+ ex %r4,Z900_G5_EX_D(%r5)
+.L_Z900_G5_4:
ipm %r2
+#if defined __s390x__
sllg %r2,%r2,34
srag %r2,%r2,62
+#else
+ sll %r2,2
+ sra %r2,30
+#endif /* ! defined __s390x__ */
br %r14
-.L_Z900_12:
+.L_Z900_G5_12:
clc 0(256,%r3),0(%r2)
- jne .L_Z900_4
+ jne .L_Z900_G5_4
la %r3,256(%r3)
la %r2,256(%r2)
- brctg %r1,.L_Z900_12
- j .L_Z900_3
-.L_Z900_15:
+ BRCTG %r1,.L_Z900_G5_12
+ j .L_Z900_G5_3
+.L_Z900_G5_15:
clc 0(1,%r3),0(%r2)
#ifdef USE_MULTIARCH
END(__memcmp_default)
@@ -62,3 +90,7 @@ END(memcmp)
libc_hidden_builtin_def (memcmp)
weak_alias (memcmp, bcmp)
#endif
+
+#undef LTGR
+#undef AGHI
+#undef BRCTG
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index c893ebc565..53dd8654d7 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -19,7 +19,8 @@ sysdep_routines += strlen strlen-vx strlen-c \
rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c \
- mempcpy
+ mempcpy \
+ memcmp memcmp-s390x
endif

ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/s390-64/multiarch/memcmp-s390x.S b/sysdeps/s390/multiarch/memcmp-s390x.S
similarity index 89%
rename from sysdeps/s390/s390-64/multiarch/memcmp-s390x.S
rename to sysdeps/s390/multiarch/memcmp-s390x.S
index 35f9bf9cf7..6321737ace 100644
--- a/sysdeps/s390/s390-64/multiarch/memcmp-s390x.S
+++ b/sysdeps/s390/multiarch/memcmp-s390x.S
@@ -1,4 +1,4 @@
-/* CPU specific memcmp implementations. 64 bit S/390 version.
+/* CPU specific memcmp implementations. 31/64 bit S/390 version.
Copyright (C) 2012-2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.

@@ -31,6 +31,10 @@

ENTRY(__memcmp_z196)
.machine "z196"
+ .machinemode "zarch_nohighgprs"
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
ltgr %r4,%r4
je .L_Z196_4
aghi %r4,-1
@@ -64,6 +68,10 @@ END(__memcmp_z196)

ENTRY(__memcmp_z10)
.machine "z10"
+ .machinemode "zarch_nohighgprs"
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
ltgr %r4,%r4
je .L_Z10_4
aghi %r4,-1
diff --git a/sysdeps/s390/s390-32/multiarch/memcmp.c b/sysdeps/s390/multiarch/memcmp.c
similarity index 100%
rename from sysdeps/s390/s390-32/multiarch/memcmp.c
rename to sysdeps/s390/multiarch/memcmp.c
diff --git a/sysdeps/s390/s390-32/memcmp.S b/sysdeps/s390/s390-32/memcmp.S
deleted file mode 100644
index f9ad0bc745..0000000000
--- a/sysdeps/s390/s390-32/memcmp.S
+++ /dev/null
@@ -1,66 +0,0 @@
-/* memcmp - compare two memory blocks. 32 bit S/390 version.
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = address of first memory area
- %r3 = address of second memory area
- %r4 = number of bytes to compare. */
-
- .text
-#ifdef USE_MULTIARCH
-ENTRY(__memcmp_default)
-#else
-ENTRY(memcmp)
-#endif
- .machine "g5"
- basr %r5,0
-.L_G5_16:
- ltr %r4,%r4
- je .L_G5_4
- ahi %r4,-1
- lr %r1,%r4
- srl %r1,8
- ltr %r1,%r1
- jne .L_G5_12
- ex %r4,.L_G5_17-.L_G5_16(%r5)
-.L_G5_4:
- ipm %r2
- sll %r2,2
- sra %r2,30
- br %r14
-.L_G5_12:
- clc 0(256,%r3),0(%r2)
- jne .L_G5_4
- la %r3,256(%r3)
- la %r2,256(%r2)
- brct %r1,.L_G5_12
- ex %r4,.L_G5_17-.L_G5_16(%r5)
- j .L_G5_4
-.L_G5_17:
- clc 0(1,%r3),0(%r2)
-#ifdef USE_MULTIARCH
-END(__memcmp_default)
-#else
-END(memcmp)
-libc_hidden_builtin_def (memcmp)
-weak_alias(memcmp, bcmp)
-#endif
diff --git a/sysdeps/s390/s390-32/multiarch/Makefile b/sysdeps/s390/s390-32/multiarch/Makefile
index 4b11e28656..82a7492eb8 100644
--- a/sysdeps/s390/s390-32/multiarch/Makefile
+++ b/sysdeps/s390/s390-32/multiarch/Makefile
@@ -1,3 +1,3 @@
ifeq ($(subdir),string)
-sysdep_routines += memcpy memcpy-s390 memcmp memcmp-s390
+sysdep_routines += memcpy memcpy-s390
endif
diff --git a/sysdeps/s390/s390-32/multiarch/memcmp-s390.S b/sysdeps/s390/s390-32/multiarch/memcmp-s390.S
deleted file mode 100644
index e53b508c98..0000000000
--- a/sysdeps/s390/s390-32/multiarch/memcmp-s390.S
+++ /dev/null
@@ -1,107 +0,0 @@
-/* CPU specific memcmp implementations. 32 bit S/390 version.
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = address of first memory area
- %r3 = address of second memory area
- %r4 = number of bytes to compare. */
-
- .text
-
-#if IS_IN (libc)
-
-ENTRY(__memcmp_z196)
- .machine "z196"
- .machinemode "zarch_nohighgprs"
- ltr %r4,%r4
- je .L_Z196_4
- ahi %r4,-1
- srlk %r1,%r4,8
- ltr %r1,%r1
- jne .L_Z196_2
-.L_Z196_3:
- exrl %r4,.L_Z196_14
-.L_Z196_4:
- ipm %r2
- sll %r2,2
- sra %r2,30
- br %r14
-.L_Z196_17:
- la %r3,256(%r3)
- la %r2,256(%r2)
- ahi %r1,-1
- je .L_Z196_3
-.L_Z196_2:
- pfd 1,512(%r3)
- pfd 1,512(%r2)
- clc 0(256,%r3),0(%r2)
- je .L_Z196_17
- ipm %r2
- sll %r2,2
- sra %r2,30
- br %r14
-.L_Z196_14:
- clc 0(1,%r3),0(%r2)
-END(__memcmp_z196)
-
-ENTRY(__memcmp_z10)
- .machine "z10"
- .machinemode "zarch_nohighgprs"
- ltr %r4,%r4
- je .L_Z10_4
- ahi %r4,-1
- lr %r1,%r4
- srl %r1,8
- cijlh %r1,0,.L_Z10_12
-.L_Z10_3:
- exrl %r4,.L_Z10_15
-.L_Z10_4:
- ipm %r2
- sll %r2,2
- sra %r2,30
- br %r14
-.L_Z10_12:
- pfd 1,512(%r3)
- pfd 1,512(%r2)
- clc 0(256,%r3),0(%r2)
- jne .L_Z10_4
- la %r3,256(%r3)
- la %r2,256(%r2)
- brct %r1,.L_Z10_12
- j .L_Z10_3
-.L_Z10_15:
- clc 0(1,%r3),0(%r2)
-END(__memcmp_z10)
-
-#endif /* IS_IN (libc) */
-
-#include "../memcmp.S"
-
-#if !IS_IN (libc)
-.globl memcmp
-.set memcmp,__memcmp_default
-.weak bcmp
-.set bcmp,__memcmp_default
-#elif defined SHARED && IS_IN (libc)
-.globl __GI_memcmp
-.set __GI_memcmp,__memcmp_default
-#endif
diff --git a/sysdeps/s390/s390-64/multiarch/Makefile b/sysdeps/s390/s390-64/multiarch/Makefile
index e4870c7ee1..8a043e3327 100644
--- a/sysdeps/s390/s390-64/multiarch/Makefile
+++ b/sysdeps/s390/s390-64/multiarch/Makefile
@@ -1,3 +1,3 @@
ifeq ($(subdir),string)
-sysdep_routines += memcpy memcpy-s390x memcmp memcmp-s390x
+sysdep_routines += memcpy memcpy-s390x
endif
diff --git a/sysdeps/s390/s390-64/multiarch/memcmp.c b/sysdeps/s390/s390-64/multiarch/memcmp.c
deleted file mode 100644
index 1e6f31806e..0000000000
--- a/sysdeps/s390/s390-64/multiarch/memcmp.c
+++ /dev/null
@@ -1,27 +0,0 @@
-/* Multiple versions of memcmp.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-#if IS_IN (libc)
-# define memcmp __redirect_memcmp
-# include <string.h>
-# undef memcmp
-# include <ifunc-resolve.h>
-
-s390_libc_ifunc (__redirect_memcmp, __memcmp, memcmp)
-weak_alias (memcmp, bcmp);
-#endif

[Stefan Liebler]

Nowadays gcc is automatically replacing a call to bcopy
with a call to memmove. Thus only old binaries will call
the s390 specific bcopy implementation.

The s390 specific implementation is using an own
implementation for memcpy in the forward case and is
relying on memmove in the backward case.

After removing the s390 specific bcopy, the common code
bcopy is used. It just performs a tail call to memmove.

ChangeLog:
* sysdeps/s390/s390-32/bcopy.S: Remove.
* sysdeps/s390/s390-64/bcopy.S: Likewise.
---
sysdeps/s390/s390-32/bcopy.S | 85 ------------------------------------
sysdeps/s390/s390-64/bcopy.S | 71 ------------------------------
2 files changed, 156 deletions(-)
delete mode 100644 sysdeps/s390/s390-32/bcopy.S
delete mode 100644 sysdeps/s390/s390-64/bcopy.S

diff --git a/sysdeps/s390/s390-32/bcopy.S b/sysdeps/s390/s390-32/bcopy.S
deleted file mode 100644
index 560e04fdee..0000000000
--- a/sysdeps/s390/s390-32/bcopy.S
+++ /dev/null
@@ -1,85 +0,0 @@
-/* bcopy -- copy a block from source to destination. S/390 version.
- This file is part of the GNU C Library.
- Copyright (C) 2000-2018 Free Software Foundation, Inc.
- Contributed by Martin Schwidefsky (***@de.ibm.com).
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* INPUT PARAMETERS
- %r2 = address of source
- %r3 = address of destination
- %r4 = number of bytes to copy. */
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
- .text
-ENTRY(__bcopy)
- ltr %r1,%r4 # zero bcopy ?
- jz .L4
- clr %r2,%r3 # check against destructive overlap
- jnl .L0
- alr %r1,%r2
- clr %r1,%r3
- jh .L7
-.L0: ahi %r4,-1 # length - 1
- lr %r1,%r4
- srl %r1,8
- ltr %r1,%r1 # < 256 bytes to move ?
- jz .L2
- chi %r1,255 # > 1MB to move ?
- jh .L5
-.L1: mvc 0(256,%r3),0(%r2) # move in 256 byte chunks
- la %r2,256(%r2)
- la %r3,256(%r3)
- brct %r1,.L1
-.L2: bras %r1,.L3 # setup base pointer for execute
- mvc 0(1,%r3),0(%r2) # instruction for execute
-.L3: ex %r4,0(%r1) # execute mvc with length ((%r4)&255)+1
-.L4: br %r14
-
- # data copies > 1MB are faster with mvcle.
-.L5: ahi %r4,1 # length + 1
- lr %r5,%r4 # source length
- lr %r4,%r2 # source address
- lr %r2,%r3 # set destination
- lr %r3,%r5 # destination length = source length
-.L6: mvcle %r2,%r4,0 # thats it, MVCLE is your friend
- jo .L6
- br %r14
-.L7: # destructive overlay, can not use mvcle
- lr %r1,%r2 # bcopy is called with source,dest
- lr %r2,%r3 # memmove with dest,source! Oh, well...
- lr %r3,%r1
- basr %r1,0
-.L8:
-#ifdef PIC
- al %r1,.L9-.L8(%r1) # get address of global offset table
- # load address of memmove
- l %r1,***@GOT(%r1)
- br %r1
-.L9: .long _GLOBAL_OFFSET_TABLE_-.L8
-#else
- al %r1,.L9-.L8(%r1) # load address of memmove
- br %r1 # jump to memmove
-.L9: .long memmove-.L8
-#endif
-
-END(__bcopy)
-
-#ifndef NO_WEAK_ALIAS
-weak_alias (__bcopy, bcopy)
-#endif
-
diff --git a/sysdeps/s390/s390-64/bcopy.S b/sysdeps/s390/s390-64/bcopy.S
deleted file mode 100644
index 806dd15d02..0000000000
--- a/sysdeps/s390/s390-64/bcopy.S
+++ /dev/null
@@ -1,71 +0,0 @@
-/* bcopy -- copy a block from source to destination. 64 bit S/390 version.
- This file is part of the GNU C Library.
- Copyright (C) 2000-2018 Free Software Foundation, Inc.
- Contributed by Martin Schwidefsky (***@de.ibm.com).
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* INPUT PARAMETERS
- %r2 = address of source
- %r3 = address of destination
- %r4 = number of bytes to copy. */
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
- .text
-ENTRY(__bcopy)
- ltgr %r1,%r4 # zero bcopy ?
- jz .L4
- clgr %r2,%r3 # check against destructive overlap
- jnl .L0
- algr %r1,%r2
- clgr %r1,%r3
- jh .L7
-.L0: aghi %r4,-1 # length - 1
- srlg %r1,%r4,8
- ltgr %r1,%r1 # < 256 bytes to move ?
- jz .L2
- cghi %r1,255 # > 1MB to move ?
- jh .L5
-.L1: mvc 0(256,%r3),0(%r2) # move in 256 byte chunks
- la %r2,256(%r2)
- la %r3,256(%r3)
- brctg %r1,.L1
-.L2: bras %r1,.L3 # setup base pointer for execute
- mvc 0(1,%r3),0(%r2) # instruction for execute
-.L3: ex %r4,0(%r1) # execute mvc with length ((%r4)&255)+1
-.L4: br %r14
- # data copies > 1MB are faster with mvcle.
-.L5: aghi %r4,1 # length + 1
- lgr %r5,%r4 # source length
- lgr %r4,%r2 # source address
- lgr %r2,%r3 # set destination
- lgr %r3,%r5 # destination length = source length
-.L6: mvcle %r2,%r4,0 # thats it, MVCLE is your friend
- jo .L6
- br %r14
-.L7: # destructive overlay, can not use mvcle
- lgr %r1,%r2 # bcopy is called with source,dest
- lgr %r2,%r3 # memmove with dest,source! Oh, well...
- lgr %r3,%r1
- jg HIDDEN_BUILTIN_JUMPTARGET(memmove)
-
-END(__bcopy)
-
-#ifndef NO_WEAK_ALIAS
-weak_alias (__bcopy, bcopy)
-#endif
-

[Stefan Liebler]

This patch moves all ifunc variants for memcmp
to sysdeps/s390/memcmp-z900.S. The configure-check/preprocessor logic
in sysdeps/s390/ifunc-memcmp.h decides if ifunc is needed at all
and which ifunc variants should be available.
E.g. if the compiler/assembler already supports z196 by default,
the older ifunc variants are not included.
If we only need the newest ifunc variant,
then we can skip ifunc at all.

Therefore the ifunc-resolvers and __libc_ifunc_impl_list are adjusted
in order to handle only the available ifunc variants.

ChangeLog:

* sysdeps/s390/ifunc-memcmp.h: New File.
* sysdeps/s390/memcmp.S: Move to ...
* sysdeps/s390/memcmp-z900.S ... here.
Move implementations from memcmp-s390x.s to here.
* sysdeps/s390/multiarch/memcmp-s390x.S: Delete File.
* sysdeps/s390/multiarch/Makefile (sysdep_routines):
Remove memcmp variants.
* sysdeps/s390/Makefile (sysdep_routines):
Add memcmp variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Adjust ifunc variants for
memcmp.
* sysdeps/s390/multiarch/memcmp.c: Move ifunc resolver
to ...
* sysdeps/s390/memcmp.c: ... here.
Adjust ifunc variants for memcmp.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-memcmp.h | 59 +++++++
.../memcmp-s390x.S => memcmp-z900.S} | 149 ++++++++++++------
sysdeps/s390/memcmp.S | 96 -----------
sysdeps/s390/{multiarch => }/memcmp.c | 26 ++-
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 16 +-
7 files changed, 204 insertions(+), 148 deletions(-)
create mode 100644 sysdeps/s390/ifunc-memcmp.h
rename sysdeps/s390/{multiarch/memcmp-s390x.S => memcmp-z900.S} (54%)
delete mode 100644 sysdeps/s390/memcmp.S
rename sysdeps/s390/{multiarch => }/memcmp.c (60%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 31ae2ce418..44a2f28bfd 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -54,5 +54,6 @@ endif
endif

ifeq ($(subdir),string)
-sysdep_routines += bzero memset memset-z900
+sysdep_routines += bzero memset memset-z900 \
+ memcmp memcmp-z900
endif
diff --git a/sysdeps/s390/ifunc-memcmp.h b/sysdeps/s390/ifunc-memcmp.h
new file mode 100644
index 0000000000..536ac455d1
--- /dev/null
+++ b/sysdeps/s390/ifunc-memcmp.h
@@ -0,0 +1,59 @@
+/* memcmp variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
+# define HAVE_MEMCMP_IFUNC 1
+#else
+# define HAVE_MEMCMP_IFUNC 0
+#endif
+
+#if defined HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
+# define MEMCMP_DEFAULT MEMCMP_Z196
+# define HAVE_MEMCMP_Z900_G5 0
+# define HAVE_MEMCMP_Z10 0
+# define HAVE_MEMCMP_Z196 1
+#elif defined HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
+# define MEMCMP_DEFAULT MEMCMP_Z10
+# define HAVE_MEMCMP_Z900_G5 0
+# define HAVE_MEMCMP_Z10 1
+# define HAVE_MEMCMP_Z196 HAVE_MEMCMP_IFUNC
+#else
+# define MEMCMP_DEFAULT MEMCMP_Z900_G5
+# define HAVE_MEMCMP_Z900_G5 1
+# define HAVE_MEMCMP_Z10 HAVE_MEMCMP_IFUNC
+# define HAVE_MEMCMP_Z196 HAVE_MEMCMP_IFUNC
+#endif
+
+#if HAVE_MEMCMP_Z900_G5
+# define MEMCMP_Z900_G5 __memcmp_default
+#else
+# define MEMCMP_Z900_G5 NULL
+#endif
+
+#if HAVE_MEMCMP_Z10
+# define MEMCMP_Z10 __memcmp_z10
+#else
+# define MEMCMP_Z10 NULL
+#endif
+
+#if HAVE_MEMCMP_Z196
+# define MEMCMP_Z196 __memcmp_z196
+#else
+# define MEMCMP_Z196 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/memcmp-s390x.S b/sysdeps/s390/memcmp-z900.S
similarity index 54%
rename from sysdeps/s390/multiarch/memcmp-s390x.S
rename to sysdeps/s390/memcmp-z900.S
index 6321737ace..c3b3677ba7 100644
--- a/sysdeps/s390/multiarch/memcmp-s390x.S
+++ b/sysdeps/s390/memcmp-z900.S
@@ -1,4 +1,4 @@
-/* CPU specific memcmp implementations. 31/64 bit S/390 version.
+/* memcmp - compare two memory blocks. 31/64 bit S/390 version.
Copyright (C) 2012-2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.

@@ -17,8 +17,9 @@
<http://www.gnu.org/licenses/>. */


-#include "sysdep.h"
+#include <sysdep.h>
#include "asm-syntax.h"
+#include <ifunc-memcmp.h>

/* INPUT PARAMETERS
%r2 = address of first memory area
@@ -27,46 +28,67 @@

.text

-#if IS_IN (libc)
-
-ENTRY(__memcmp_z196)
- .machine "z196"
- .machinemode "zarch_nohighgprs"
-# if !defined __s390x__
- llgfr %r4,%r4
-# endif /* !defined __s390x__ */
- ltgr %r4,%r4
- je .L_Z196_4
- aghi %r4,-1
+#if HAVE_MEMCMP_Z900_G5
+# if defined __s390x__
+# define LTGR ltgr
+# define AGHI aghi
+# define BRCTG brctg
+# else
+# define LTGR ltr
+# define AGHI ahi
+# define BRCTG brct
+# endif /* ! defined __s390x__ */
+ENTRY(MEMCMP_Z900_G5)
+# if defined __s390x__
+ .machine "z900"
+# else
+ .machine "g5"
+ basr %r5,0
+.L_Z900_G5_16:
+# define Z900_G5_EX_D .L_Z900_G5_15-.L_Z900_G5_16
+# endif /* ! defined __s390x__ */
+ LTGR %r4,%r4
+ je .L_Z900_G5_4
+ AGHI %r4,-1
+# if defined __s390x__
srlg %r1,%r4,8
- ltgr %r1,%r1
- jne .L_Z196_2
-.L_Z196_3:
- exrl %r4,.L_Z196_14
-.L_Z196_4:
+ larl %r5,.L_Z900_G5_15
+# define Z900_G5_EX_D 0
+# else
+ lr %r1,%r4
+ srl %r1,8
+# endif /* ! defined __s390x__ */
+ LTGR %r1,%r1
+ jne .L_Z900_G5_12
+.L_Z900_G5_3:
+ ex %r4,Z900_G5_EX_D(%r5)
+.L_Z900_G5_4:
ipm %r2
+# if defined __s390x__
sllg %r2,%r2,34
srag %r2,%r2,62
+# else
+ sll %r2,2
+ sra %r2,30
+# endif /* ! defined __s390x__ */
br %r14
-.L_Z196_17:
+.L_Z900_G5_12:
+ clc 0(256,%r3),0(%r2)
+ jne .L_Z900_G5_4
la %r3,256(%r3)
la %r2,256(%r2)
- aghi %r1,-1
- je .L_Z196_3
-.L_Z196_2:
- pfd 1,512(%r3)
- pfd 1,512(%r2)
- clc 0(256,%r3),0(%r2)
- je .L_Z196_17
- ipm %r2
- sllg %r2,%r2,34
- srag %r2,%r2,62
- br %r14
-.L_Z196_14:
+ BRCTG %r1,.L_Z900_G5_12
+ j .L_Z900_G5_3
+.L_Z900_G5_15:
clc 0(1,%r3),0(%r2)
-END(__memcmp_z196)
+END(MEMCMP_Z900_G5)
+# undef LTGR
+# undef AGHI
+# undef BRCTG
+#endif /* HAVE_MEMCMP_Z900_G5 */

-ENTRY(__memcmp_z10)
+#if HAVE_MEMCMP_Z10
+ENTRY(MEMCMP_Z10)
.machine "z10"
.machinemode "zarch_nohighgprs"
# if !defined __s390x__
@@ -95,18 +117,57 @@ ENTRY(__memcmp_z10)
j .L_Z10_3
.L_Z10_15:
clc 0(1,%r3),0(%r2)
-END(__memcmp_z10)
+END(MEMCMP_Z10)
+#endif /* HAVE_MEMCMP_Z10 */

-#endif /* IS_IN (libc) */
+#if HAVE_MEMCMP_Z196
+ENTRY(MEMCMP_Z196)
+ .machine "z196"
+ .machinemode "zarch_nohighgprs"
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+ ltgr %r4,%r4
+ je .L_Z196_4
+ aghi %r4,-1
+ srlg %r1,%r4,8
+ ltgr %r1,%r1
+ jne .L_Z196_2
+.L_Z196_3:
+ exrl %r4,.L_Z196_14
+.L_Z196_4:
+ ipm %r2
+ sllg %r2,%r2,34
+ srag %r2,%r2,62
+ br %r14
+.L_Z196_17:
+ la %r3,256(%r3)
+ la %r2,256(%r2)
+ aghi %r1,-1
+ je .L_Z196_3
+.L_Z196_2:
+ pfd 1,512(%r3)
+ pfd 1,512(%r2)
+ clc 0(256,%r3),0(%r2)
+ je .L_Z196_17
+ ipm %r2
+ sllg %r2,%r2,34
+ srag %r2,%r2,62
+ br %r14
+.L_Z196_14:
+ clc 0(1,%r3),0(%r2)
+END(MEMCMP_Z196)
+#endif /* HAVE_MEMCMP_Z196 */

-#include "../memcmp.S"
+#if ! HAVE_MEMCMP_IFUNC
+/* If we don't use ifunc, define an alias for memcmp here.
+ Otherwise see sysdeps/s390/memcmp.c. */
+strong_alias (MEMCMP_DEFAULT, memcmp)
+weak_alias (memcmp, bcmp)
+#endif

-#if !IS_IN (libc)
-.globl memcmp
-.set memcmp,__memcmp_default
-.weak bcmp
-.set bcmp,__memcmp_default
-#elif defined SHARED && IS_IN (libc)
-.globl __GI_memcmp
-.set __GI_memcmp,__memcmp_default
+#if defined SHARED && IS_IN (libc)
+/* Defines the internal symbols.
+ Compare to libc_hidden_builtin_def (memcmp) in string/memcmp.c. */
+strong_alias (MEMCMP_DEFAULT, __GI_memcmp)
#endif
diff --git a/sysdeps/s390/memcmp.S b/sysdeps/s390/memcmp.S
deleted file mode 100644
index 751293a99e..0000000000
--- a/sysdeps/s390/memcmp.S
+++ /dev/null
@@ -1,96 +0,0 @@
-/* memcmp - compare two memory blocks. 31/64 bit S/390 version.
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include <sysdep.h>
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = address of first memory area
- %r3 = address of second memory area
- %r4 = number of bytes to compare. */
-
- .text
-#if defined __s390x__
-# define LTGR ltgr
-# define AGHI aghi
-# define BRCTG brctg
-#else
-# define LTGR ltr
-# define AGHI ahi
-# define BRCTG brct
-#endif /* ! defined __s390x__ */
-
-#ifdef USE_MULTIARCH
-ENTRY(__memcmp_default)
-#else
-ENTRY(memcmp)
-#endif
-#if defined __s390x__
- .machine "z900"
-#else
- .machine "g5"
- basr %r5,0
-.L_Z900_G5_16:
-# define Z900_G5_EX_D .L_Z900_G5_15-.L_Z900_G5_16
-#endif /* ! defined __s390x__ */
- LTGR %r4,%r4
- je .L_Z900_G5_4
- AGHI %r4,-1
-#if defined __s390x__
- srlg %r1,%r4,8
- larl %r5,.L_Z900_G5_15
-# define Z900_G5_EX_D 0
-#else
- lr %r1,%r4
- srl %r1,8
-#endif /* ! defined __s390x__ */
- LTGR %r1,%r1
- jne .L_Z900_G5_12
-.L_Z900_G5_3:
- ex %r4,Z900_G5_EX_D(%r5)
-.L_Z900_G5_4:
- ipm %r2
-#if defined __s390x__
- sllg %r2,%r2,34
- srag %r2,%r2,62
-#else
- sll %r2,2
- sra %r2,30
-#endif /* ! defined __s390x__ */
- br %r14
-.L_Z900_G5_12:
- clc 0(256,%r3),0(%r2)
- jne .L_Z900_G5_4
- la %r3,256(%r3)
- la %r2,256(%r2)
- BRCTG %r1,.L_Z900_G5_12
- j .L_Z900_G5_3
-.L_Z900_G5_15:
- clc 0(1,%r3),0(%r2)
-#ifdef USE_MULTIARCH
-END(__memcmp_default)
-#else
-END(memcmp)
-libc_hidden_builtin_def (memcmp)
-weak_alias (memcmp, bcmp)
-#endif
-
-#undef LTGR
-#undef AGHI
-#undef BRCTG
diff --git a/sysdeps/s390/multiarch/memcmp.c b/sysdeps/s390/memcmp.c
similarity index 60%
rename from sysdeps/s390/multiarch/memcmp.c
rename to sysdeps/s390/memcmp.c
index 1e6f31806e..952ff6af73 100644
--- a/sysdeps/s390/multiarch/memcmp.c
+++ b/sysdeps/s390/memcmp.c
@@ -16,12 +16,34 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if IS_IN (libc)
+#include <ifunc-memcmp.h>
+#if HAVE_MEMCMP_IFUNC
# define memcmp __redirect_memcmp
# include <string.h>
# undef memcmp
# include <ifunc-resolve.h>

-s390_libc_ifunc (__redirect_memcmp, __memcmp, memcmp)
+# if HAVE_MEMCMP_Z900_G5
+extern __typeof (__redirect_memcmp) MEMCMP_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_MEMCMP_Z10
+extern __typeof (__redirect_memcmp) MEMCMP_Z10 attribute_hidden;
+# endif
+
+# if HAVE_MEMCMP_Z196
+extern __typeof (__redirect_memcmp) MEMCMP_Z196 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect_memcmp, memcmp,
+ ({
+ s390_libc_ifunc_init ();
+ (HAVE_MEMCMP_Z196 && S390_IS_Z196 (stfle_bits))
+ ? MEMCMP_Z196
+ : (HAVE_MEMCMP_Z10 && S390_IS_Z10 (stfle_bits))
+ ? MEMCMP_Z10
+ : MEMCMP_DEFAULT;
+ })
+ )
weak_alias (memcmp, bcmp);
#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 53dd8654d7..c893ebc565 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -19,8 +19,7 @@ sysdep_routines += strlen strlen-vx strlen-c \
rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c \
- mempcpy \
- memcmp memcmp-s390x
+ mempcpy
endif

ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 253f36045b..2e57d01abc 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -22,6 +22,7 @@
#include <ifunc-impl-list.h>
#include <ifunc-resolve.h>
#include <ifunc-memset.h>
+#include <ifunc-memcmp.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -78,12 +79,21 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_MEMSET_IFUNC */

+#if HAVE_MEMCMP_IFUNC
IFUNC_IMPL (i, name, memcmp,
+# if HAVE_MEMCMP_Z196
IFUNC_IMPL_ADD (array, i, memcmp,
- S390_IS_Z196 (stfle_bits), __memcmp_z196)
+ S390_IS_Z196 (stfle_bits), MEMCMP_Z196)
+# endif
+# if HAVE_MEMCMP_Z10
IFUNC_IMPL_ADD (array, i, memcmp,
- S390_IS_Z10 (stfle_bits), __memcmp_z10)
- IFUNC_IMPL_ADD (array, i, memcmp, 1, __memcmp_default))
+ S390_IS_Z10 (stfle_bits), MEMCMP_Z10)
+# endif
+# if HAVE_MEMCMP_Z900_G5
+ IFUNC_IMPL_ADD (array, i, memcmp, 1, MEMCMP_Z900_G5)
+# endif
+ )
+#endif /* HAVE_MEMCMP_IFUNC */

#ifdef SHARED

[Stefan Liebler]

The implementation of memcpy/mempcpy for s390-32 (31bit)
and s390-64 (64bit) is nearly the same.
This patch unifies it for maintability reasons.

__mem[p]cpy_z10 and __mem[p]cpy_z196 differs between 31 and 64bit:
-31bit needs .machinemode "zarch_nohighgprs" and llgfr %r4,%r4
-lr vs lgr; lgr can be also used on 31bit as this ifunc variant
is only called if we are on a zarch machine.

__mem[p]cpy_default differs between 31 and 64bit:
-Some 31bit vs 64bit instructions (e.g. ltr vs ltgr.
Solved with 31/64 specific instruction macros).
-The address of mvc instruction is setup in different ways
(larl vs bras). Solved with #if defined __s390x__.

__memcpy_mvcle differs between 31 and 64bit:
-lr vs lgr; ahi vs aghi;
Solved with 31/64bit specific instruction macros.

Otherwise 31/64bit implementation has the same structure of the code.

ChangeLog:

* sysdeps/s390/s390-64/memcpy.S: Move to ...
* sysdeps/s390/memcpy.S: ... here.
Adjust to be usable for 31/64bit.
* sysdeps/s390/s390-32/memcpy.S: Delete File.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add memcpy.
* sysdeps/s390/s390-32/multiarch/Makefile: Delete file.
* sysdeps/s390/s390-64/multiarch/Makefile: Likewise.
* sysdeps/s390/s390-64/multiarch/memcpy-s390x.S: Move to ...
* sysdeps/s390/multiarch/memcpy-s390x.S: ... here.
Adjust to be usable for 31/64bit.
* sysdeps/s390/s390-32/multiarch/memcpy-s390.S: Delete File.
* sysdeps/s390/s390-64/multiarch/memcpy.c: Move to ...
* sysdeps/s390/multiarch/memcpy.c: ... here.
* sysdeps/s390/s390-32/multiarch/memcpy.c: Delete File.
---
sysdeps/s390/{s390-64 => }/memcpy.S | 98 ++++++++++----
sysdeps/s390/multiarch/Makefile | 2 +-
.../{s390-64 => }/multiarch/memcpy-s390x.S | 12 +-
sysdeps/s390/{s390-32 => }/multiarch/memcpy.c | 0
sysdeps/s390/s390-32/memcpy.S | 89 ------------
sysdeps/s390/s390-32/multiarch/Makefile | 3 -
sysdeps/s390/s390-32/multiarch/memcpy-s390.S | 128 ------------------
sysdeps/s390/s390-64/multiarch/Makefile | 3 -
sysdeps/s390/s390-64/multiarch/memcpy.c | 27 ----
9 files changed, 81 insertions(+), 281 deletions(-)
rename sysdeps/s390/{s390-64 => }/memcpy.S (52%)
rename sysdeps/s390/{s390-64 => }/multiarch/memcpy-s390x.S (89%)
rename sysdeps/s390/{s390-32 => }/multiarch/memcpy.c (100%)
delete mode 100644 sysdeps/s390/s390-32/memcpy.S
delete mode 100644 sysdeps/s390/s390-32/multiarch/Makefile
delete mode 100644 sysdeps/s390/s390-32/multiarch/memcpy-s390.S
delete mode 100644 sysdeps/s390/s390-64/multiarch/Makefile
delete mode 100644 sysdeps/s390/s390-64/multiarch/memcpy.c

diff --git a/sysdeps/s390/s390-64/memcpy.S b/sysdeps/s390/memcpy.S
similarity index 52%
rename from sysdeps/s390/s390-64/memcpy.S
rename to sysdeps/s390/memcpy.S
index 2e5490df23..2a6c6b7503 100644
--- a/sysdeps/s390/s390-64/memcpy.S
+++ b/sysdeps/s390/memcpy.S
@@ -1,4 +1,4 @@
-/* memcpy - copy a block from source to destination. 64 bit S/390 version.
+/* memcpy - copy a block from source to destination. 31/64 bit S/390 version.
Copyright (C) 2012-2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.

@@ -25,13 +25,31 @@
%r3 = address of source memory area
%r4 = number of bytes to copy. */

-
.text
+
+#if defined __s390x__
+# define LTGR ltgr
+# define CGHI cghi
+# define LGR lgr
+# define AGHI aghi
+# define BRCTG brctg
+#else
+# define LTGR ltr
+# define CGHI chi
+# define LGR lr
+# define AGHI ahi
+# define BRCTG brct
+#endif /* ! defined __s390x__ */
+
ENTRY(__mempcpy)
+#if defined __s390x__
.machine "z900"
- lgr %r1,%r2 # Use as dest
+#else
+ .machine "g5"
+#endif /* ! defined __s390x__ */
+ LGR %r1,%r2 # Use as dest
la %r2,0(%r4,%r2) # Return dest + n
- j .L_Z900_start
+ j .L_Z900_G5_start
END(__mempcpy)
#ifndef USE_MULTIARCH
libc_hidden_def (__mempcpy)
@@ -40,30 +58,46 @@ libc_hidden_builtin_def (mempcpy)
#endif

ENTRY(memcpy)
+#if defined __s390x__
.machine "z900"
- lgr %r1,%r2 # r1: Use as dest ; r2: Return dest
-.L_Z900_start:
- ltgr %r4,%r4
- je .L_Z900_4
- aghi %r4,-1
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z900_13
-.L_Z900_3:
- larl %r5,.L_Z900_15
- ex %r4,0(%r5)
-.L_Z900_4:
+#else
+ .machine "g5"
+#endif /* ! defined __s390x__ */
+ LGR %r1,%r2 # r1: Use as dest ; r2: Return dest
+.L_Z900_G5_start:
+ LTGR %r4,%r4
+ je .L_Z900_G5_4
+ AGHI %r4,-1
+#if defined __s390x__
+ srlg %r5,%r4,8
+#else
+ lr %r5,%r4
+ srl %r5,8
+#endif /* ! defined __s390x__ */
+ LTGR %r5,%r5
+ jne .L_Z900_G5_13
+.L_Z900_G5_3:
+#if defined __s390x__
+ larl %r5,.L_Z900_G5_15
+# define Z900_G5_EX_D 0
+#else
+ basr %r5,0
+.L_Z900_G5_14:
+# define Z900_G5_EX_D .L_Z900_G5_15-.L_Z900_G5_14
+#endif /* ! defined __s390x__ */
+ ex %r4,Z900_G5_EX_D(%r5)
+.L_Z900_G5_4:
br %r14
-.L_Z900_13:
- cghi %r5,4096 # Switch to mvcle for copies >1MB
+.L_Z900_G5_13:
+ CGHI %r5,4096 # Switch to mvcle for copies >1MB
jh __memcpy_mvcle
-.L_Z900_12:
+.L_Z900_G5_12:
mvc 0(256,%r1),0(%r3)
la %r1,256(%r1)
la %r3,256(%r3)
- brctg %r5,.L_Z900_12
- j .L_Z900_3
-.L_Z900_15:
+ BRCTG %r5,.L_Z900_G5_12
+ j .L_Z900_G5_3
+.L_Z900_G5_15:
mvc 0(1,%r1),0(%r3)
END(memcpy)
#ifndef USE_MULTIARCH
@@ -74,15 +108,21 @@ ENTRY(__memcpy_mvcle)
# Using as standalone function will result in unexpected
# results since the length field is incremented by 1 in order to
# compensate the changes already done in the functions above.
- lgr %r0,%r2 # backup return dest [ + n ]
- aghi %r4,1 # length + 1
- lgr %r5,%r4 # source length
- lgr %r4,%r3 # source address
- lgr %r2,%r1 # destination address
- lgr %r3,%r5 # destination length = source length
+ LGR %r0,%r2 # backup return dest [ + n ]
+ AGHI %r4,1 # length + 1
+ LGR %r5,%r4 # source length
+ LGR %r4,%r3 # source address
+ LGR %r2,%r1 # destination address
+ LGR %r3,%r5 # destination length = source length
.L_MVCLE_1:
mvcle %r2,%r4,0 # thats it, MVCLE is your friend
jo .L_MVCLE_1
- lgr %r2,%r0 # return destination address
+ LGR %r2,%r0 # return destination address
br %r14
END(__memcpy_mvcle)
+
+#undef LTGR
+#undef CGHI
+#undef LGR
+#undef AGHI
+#undef BRCTG
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index c893ebc565..3cbd5fad69 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -19,7 +19,7 @@ sysdep_routines += strlen strlen-vx strlen-c \
rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c \
- mempcpy
+ mempcpy memcpy memcpy-s390x
endif

ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/s390-64/multiarch/memcpy-s390x.S b/sysdeps/s390/multiarch/memcpy-s390x.S
similarity index 89%
rename from sysdeps/s390/s390-64/multiarch/memcpy-s390x.S
rename to sysdeps/s390/multiarch/memcpy-s390x.S
index 6d60a70834..b38caac72b 100644
--- a/sysdeps/s390/s390-64/multiarch/memcpy-s390x.S
+++ b/sysdeps/s390/multiarch/memcpy-s390x.S
@@ -1,4 +1,4 @@
-/* CPU specific memcpy implementations. 64 bit S/390 version.
+/* CPU specific memcpy implementations. 31/64 bit S/390 version.
Copyright (C) 2012-2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.

@@ -31,6 +31,7 @@

ENTRY(____mempcpy_z196)
.machine "z196"
+ .machinemode "zarch_nohighgprs"
lgr %r1,%r2 # Use as dest
la %r2,0(%r4,%r2) # Return dest + n
j .L_Z196_start
@@ -38,8 +39,12 @@ END(____mempcpy_z196)

ENTRY(__memcpy_z196)
.machine "z196"
+ .machinemode "zarch_nohighgprs"
lgr %r1,%r2 # r1: Use as dest ; r2: Return dest
.L_Z196_start:
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
ltgr %r4,%r4
je .L_Z196_4
aghi %r4,-1
@@ -68,6 +73,7 @@ END(__memcpy_z196)

ENTRY(____mempcpy_z10)
.machine "z10"
+ .machinemode "zarch_nohighgprs"
lgr %r1,%r2 # Use as dest
la %r2,0(%r4,%r2) # Return dest + n
j .L_Z10_start
@@ -75,8 +81,12 @@ END(____mempcpy_z10)

ENTRY(__memcpy_z10)
.machine "z10"
+ .machinemode "zarch_nohighgprs"
lgr %r1,%r2 # r1: Use as dest ; r2: Return dest
.L_Z10_start:
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
cgije %r4,0,.L_Z10_4
aghi %r4,-1
srlg %r5,%r4,8
diff --git a/sysdeps/s390/s390-32/multiarch/memcpy.c b/sysdeps/s390/multiarch/memcpy.c
similarity index 100%
rename from sysdeps/s390/s390-32/multiarch/memcpy.c
rename to sysdeps/s390/multiarch/memcpy.c
diff --git a/sysdeps/s390/s390-32/memcpy.S b/sysdeps/s390/s390-32/memcpy.S
deleted file mode 100644
index 493cc18aba..0000000000
--- a/sysdeps/s390/s390-32/memcpy.S
+++ /dev/null
@@ -1,89 +0,0 @@
-/* memcpy - copy a block from source to destination. S/390 version.
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = address of destination memory area
- %r3 = address of source memory area
- %r4 = number of bytes to copy. */
-
- .text
-ENTRY(__mempcpy)
- .machine "g5"
- lr %r1,%r2 # Use as dest
- la %r2,0(%r4,%r2) # Return dest + n
- j .L_G5_start
-END(__mempcpy)
-#ifndef USE_MULTIARCH
-libc_hidden_def (__mempcpy)
-weak_alias (__mempcpy, mempcpy)
-libc_hidden_builtin_def (mempcpy)
-#endif
-
-ENTRY(memcpy)
- .machine "g5"
- lr %r1,%r2 # r1: Use as dest ; r2: Return dest
-.L_G5_start:
- ltr %r4,%r4
- je .L_G5_99
- ahi %r4,-1
- lr %r5,%r4
- srl %r5,8
- ltr %r5,%r5
- jne .L_G5_13
-.L_G5_4:
- basr %r5,0
-.L_G5_16:
- ex %r4,.L_G5_17-.L_G5_16(%r5)
-.L_G5_99:
- br %r14
-.L_G5_13:
- chi %r5,4096 # Switch to mvcle for copies >1MB
- jh __memcpy_mvcle
-.L_G5_12:
- mvc 0(256,%r1),0(%r3)
- la %r1,256(%r1)
- la %r3,256(%r3)
- brct %r5,.L_G5_12
- j .L_G5_4
-.L_G5_17:
- mvc 0(1,%r1),0(%r3)
-END(memcpy)
-#ifndef USE_MULTIARCH
-libc_hidden_builtin_def (memcpy)
-#endif
-
-ENTRY(__memcpy_mvcle)
- # Using as standalone function will result in unexpected
- # results since the length field is incremented by 1 in order to
- # compensate the changes already done in the functions above.
- lr %r0,%r2 # backup return dest [ + n ]
- ahi %r4,1 # length + 1
- lr %r5,%r4 # source length
- lr %r4,%r3 # source address
- lr %r2,%r1 # destination address
- lr %r3,%r5 # destination length = source length
-.L_MVCLE_1:
- mvcle %r2,%r4,0 # thats it, MVCLE is your friend
- jo .L_MVCLE_1
- lr %r2,%r0 # return destination address
- br %r14
-END(__memcpy_mvcle)
diff --git a/sysdeps/s390/s390-32/multiarch/Makefile b/sysdeps/s390/s390-32/multiarch/Makefile
deleted file mode 100644
index 82a7492eb8..0000000000
--- a/sysdeps/s390/s390-32/multiarch/Makefile
+++ /dev/null
@@ -1,3 +0,0 @@
-ifeq ($(subdir),string)
-sysdep_routines += memcpy memcpy-s390
-endif
diff --git a/sysdeps/s390/s390-32/multiarch/memcpy-s390.S b/sysdeps/s390/s390-32/multiarch/memcpy-s390.S
deleted file mode 100644
index aad13bd07c..0000000000
--- a/sysdeps/s390/s390-32/multiarch/memcpy-s390.S
+++ /dev/null
@@ -1,128 +0,0 @@
-/* CPU specific memcpy implementations. 32 bit S/390 version.
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = target operands address
- %r3 = source operands address
- %r4 = number of bytes to copy. */
-
- .text
-
-#if defined SHARED && IS_IN (libc)
-
-ENTRY(____mempcpy_z196)
- .machine "z196"
- .machinemode "zarch_nohighgprs"
- lr %r1,%r2 # Use as dest
- la %r2,0(%r4,%r2) # Return dest + n
- j .L_Z196_start
-END(____mempcpy_z196)
-
-ENTRY(__memcpy_z196)
- .machine "z196"
- .machinemode "zarch_nohighgprs"
- lr %r1,%r2 # r1: Use as dest ; r2: Return dest
-.L_Z196_start:
- llgfr %r4,%r4
- ltgr %r4,%r4
- je .L_Z196_4
- aghi %r4,-1
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z196_5
-.L_Z196_3:
- exrl %r4,.L_Z196_14
-.L_Z196_4:
- br %r14
-.L_Z196_5:
- cgfi %r5,262144 # Switch to mvcle for copies >64MB
- jh __memcpy_mvcle
-.L_Z196_2:
- pfd 1,768(%r3)
- pfd 2,768(%r1)
- mvc 0(256,%r1),0(%r3)
- aghi %r5,-1
- la %r1,256(%r1)
- la %r3,256(%r3)
- jne .L_Z196_2
- j .L_Z196_3
-.L_Z196_14:
- mvc 0(1,%r1),0(%r3)
-END(__memcpy_z196)
-
-ENTRY(____mempcpy_z10)
- .machine "z10"
- .machinemode "zarch_nohighgprs"
- lr %r1,%r2 # Use as dest
- la %r2,0(%r4,%r2) # Return dest + n
- j .L_Z10_start
-END(____mempcpy_z10)
-
-ENTRY(__memcpy_z10)
- .machine "z10"
- .machinemode "zarch_nohighgprs"
- lr %r1,%r2 # r1: Use as dest ; r2: Return dest
-.L_Z10_start:
- llgfr %r4,%r4
- cgije %r4,0,.L_Z10_4
- aghi %r4,-1
- srlg %r5,%r4,8
- cgijlh %r5,0,.L_Z10_13
-.L_Z10_3:
- exrl %r4,.L_Z10_15
-.L_Z10_4:
- br %r14
-.L_Z10_13:
- cgfi %r5,65535 # Switch to mvcle for copies >16MB
- jh __memcpy_mvcle
-.L_Z10_12:
- pfd 1,768(%r3)
- pfd 2,768(%r1)
- mvc 0(256,%r1),0(%r3)
- la %r1,256(%r1)
- la %r3,256(%r3)
- brctg %r5,.L_Z10_12
- j .L_Z10_3
-.L_Z10_15:
- mvc 0(1,%r1),0(%r3)
-END(__memcpy_z10)
-
-# define __mempcpy ____mempcpy_default
-#endif /* SHARED && IS_IN (libc) */
-
-#define memcpy __memcpy_default
-#include "../memcpy.S"
-#undef memcpy
-
-#if defined SHARED && IS_IN (libc)
-.globl __GI_memcpy
-.set __GI_memcpy,__memcpy_default
-.globl __GI_mempcpy
-.set __GI_mempcpy,____mempcpy_default
-.globl __GI___mempcpy
-.set __GI___mempcpy,____mempcpy_default
-#else
-.globl memcpy
-.set memcpy,__memcpy_default
-.weak mempcpy
-.set mempcpy,__mempcpy
-#endif
diff --git a/sysdeps/s390/s390-64/multiarch/Makefile b/sysdeps/s390/s390-64/multiarch/Makefile
deleted file mode 100644
index 8a043e3327..0000000000
--- a/sysdeps/s390/s390-64/multiarch/Makefile
+++ /dev/null
@@ -1,3 +0,0 @@
-ifeq ($(subdir),string)
-sysdep_routines += memcpy memcpy-s390x
-endif
diff --git a/sysdeps/s390/s390-64/multiarch/memcpy.c b/sysdeps/s390/s390-64/multiarch/memcpy.c
deleted file mode 100644
index c9577a854a..0000000000
--- a/sysdeps/s390/s390-64/multiarch/memcpy.c
+++ /dev/null
@@ -1,27 +0,0 @@
-/* Multiple versions of memcpy.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* In the static lib memcpy is needed before the reloc is resolved. */
-#if defined SHARED && IS_IN (libc)
-# define memcpy __redirect_memcpy
-# include <string.h>
-# undef memcpy
-# include <ifunc-resolve.h>
-
-s390_libc_ifunc (__redirect_memcpy, __memcpy, memcpy)
-#endif

[Stefan Liebler]

The new vector variant of strstr is using the common code
implementation, but instead of calling the default
str* / mem* functions, the vector variants are called.

ChangeLog:

* sysdeps/s390/Makefile (sysdep_routines): Add strstr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc variants for strstr.
* sysdeps/s390/ifunc-strstr.h: New file.
* sysdeps/s390/strstr.c: Likewise.
* sysdeps/s390/strstr-c.c: Likewise.
* sysdeps/s390/strstr-vx.c: Likewise.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strstr.h | 52 ++++++++++++++++++++++++
sysdeps/s390/multiarch/ifunc-impl-list.c | 13 ++++++
sysdeps/s390/strstr-c.c | 32 +++++++++++++++
sysdeps/s390/strstr-vx.c | 52 ++++++++++++++++++++++++
sysdeps/s390/strstr.c | 40 ++++++++++++++++++
6 files changed, 191 insertions(+), 1 deletion(-)
create mode 100644 sysdeps/s390/ifunc-strstr.h
create mode 100644 sysdeps/s390/strstr-c.c
create mode 100644 sysdeps/s390/strstr-vx.c
create mode 100644 sysdeps/s390/strstr.c

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 859f36f249..c886fe31f3 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -57,5 +57,6 @@ ifeq ($(subdir),string)
sysdep_routines += bzero memset memset-z900 \
memcmp memcmp-z900 \
mempcpy memcpy memcpy-z900 \
- memmove memmove-c
+ memmove memmove-c \
+ strstr strstr-vx strstr-c
endif
diff --git a/sysdeps/s390/ifunc-strstr.h b/sysdeps/s390/ifunc-strstr.h
new file mode 100644
index 0000000000..e6ccfd4e44
--- /dev/null
+++ b/sysdeps/s390/ifunc-strstr.h
@@ -0,0 +1,52 @@
+/* strstr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRSTR_IFUNC 1
+#else
+# define HAVE_STRSTR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRSTR_IFUNC_AND_VX_SUPPORT HAVE_STRSTR_IFUNC
+#else
+# define HAVE_STRSTR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRSTR_DEFAULT STRSTR_Z13
+# define HAVE_STRSTR_C 0
+# define HAVE_STRSTR_Z13 1
+#else
+# define STRSTR_DEFAULT STRSTR_C
+# define HAVE_STRSTR_C 1
+# define HAVE_STRSTR_Z13 HAVE_STRSTR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRSTR_C
+# define STRSTR_C __strstr_c
+#else
+# define STRSTR_C NULL
+#endif
+
+#if HAVE_STRSTR_Z13
+# define STRSTR_Z13 __strstr_vx
+#else
+# define STRSTR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index c05c63e006..14727f8fef 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -24,6 +24,7 @@
#include <ifunc-memset.h>
#include <ifunc-memcmp.h>
#include <ifunc-memcpy.h>
+#include <ifunc-strstr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -138,6 +139,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_MEMMOVE_IFUNC */

+#if HAVE_STRSTR_IFUNC
+ IFUNC_IMPL (i, name, strstr,
+# if HAVE_STRSTR_Z13
+ IFUNC_IMPL_ADD (array, i, strstr,
+ dl_hwcap & HWCAP_S390_VX, STRSTR_Z13)
+# endif
+# if HAVE_STRSTR_C
+ IFUNC_IMPL_ADD (array, i, strstr, 1, STRSTR_C)
+# endif
+ )
+#endif /* HAVE_STRSTR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
diff --git a/sysdeps/s390/strstr-c.c b/sysdeps/s390/strstr-c.c
new file mode 100644
index 0000000000..53717bfb27
--- /dev/null
+++ b/sysdeps/s390/strstr-c.c
@@ -0,0 +1,32 @@
+/* Default strstr implementation for S/390.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <ifunc-strstr.h>
+
+#if HAVE_STRSTR_C
+# if HAVE_STRSTR_IFUNC
+# define STRSTR STRSTR_C
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strstr_c, __GI_strstr, __strstr_c);
+# endif
+# endif
+
+# include <string/strstr.c>
+#endif
diff --git a/sysdeps/s390/strstr-vx.c b/sysdeps/s390/strstr-vx.c
new file mode 100644
index 0000000000..effae9d5eb
--- /dev/null
+++ b/sysdeps/s390/strstr-vx.c
@@ -0,0 +1,52 @@
+/* Default strstr implementation with vector string functions for S/390.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <ifunc-strstr.h>
+
+#if HAVE_STRSTR_Z13
+# if HAVE_STRSTR_IFUNC
+# define STRSTR STRSTR_Z13
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# if HAVE_STRSTR_C
+# define libc_hidden_builtin_def(name)
+# else
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strstr_vx, __GI_strstr, __strstr_vx);
+# endif
+# endif
+# endif
+
+# include <string.h>
+
+# ifdef USE_MULTIARCH
+extern __typeof (strchr) __strchr_vx attribute_hidden;
+# define strchr __strchr_vx
+
+extern __typeof (strlen) __strlen_vx attribute_hidden;
+# define strlen __strlen_vx
+
+extern __typeof (__strnlen) __strnlen_vx attribute_hidden;
+# define __strnlen __strnlen_vx
+
+extern __typeof (memcmp) __memcmp_z196 attribute_hidden;
+# define memcmp __memcmp_z196
+# endif
+
+# include <string/strstr.c>
+#endif
diff --git a/sysdeps/s390/strstr.c b/sysdeps/s390/strstr.c
new file mode 100644
index 0000000000..f8432349a7
--- /dev/null
+++ b/sysdeps/s390/strstr.c
@@ -0,0 +1,40 @@
+/* Multiple versions of strstr.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <ifunc-strstr.h>
+
+#if HAVE_STRSTR_IFUNC
+# define strstr __redirect_strstr
+# include <string.h>
+# include <ifunc-resolve.h>
+# undef strstr
+
+# if HAVE_STRSTR_C
+extern __typeof (__redirect_strstr) STRSTR_C attribute_hidden;
+# endif
+
+# if HAVE_STRSTR_Z13
+extern __typeof (__redirect_strstr) STRSTR_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect_strstr, strstr,
+ (HAVE_STRSTR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRSTR_Z13
+ : STRSTR_DEFAULT
+ )
+#endif

[Stefan Liebler]

This patch introduces a z13 specific ifunc variant for memmove.
As the common code implementation, it checks if we can copy from
the beginning to the end - with z196 memcpy implementation - or
if we have to copy from the end to the beginning.
The latter case is done by using vector load/store instructions.

If vector instructions are not available, the common-code is
used as fallback. Therefore it is implemented in memmove-c with
a different name.
Furthermore the ifunc logic decides if we need the common-code
implementation at all. If vector instructions are supported
due to the minimum architecture level set we can skip the
common-code ifunc variant.

ChangeLog:

* sysdeps/s390/Makefile (sysdep_routines): Add memmove-c.
* sysdeps/s390/ifunc-memcpy.h (HAVE_MEMMOVE_IFUNC,
HAVE_MEMMOVE_IFUNC_AND_VX_SUPPORT, MEMMOVE_DEFAULT,
HAVE_MEMMOVE_C, MEMMOVE_C, HAVE_MEMMOVE_Z13, MEMMOVE_Z13):
New defines.
* sysdeps/s390/memcpy-z900.S: Add z13 memmove implementation.
* sysdeps/s390/memmove-c.c: New file.
* sysdeps/s390/memmove.c: Likewise.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc variants for memmove.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-memcpy.h | 35 ++++++++++
sysdeps/s390/memcpy-z900.S | 87 ++++++++++++++++++++++++
sysdeps/s390/memmove-c.c | 37 ++++++++++
sysdeps/s390/memmove.c | 44 ++++++++++++
sysdeps/s390/multiarch/ifunc-impl-list.c | 12 ++++
6 files changed, 217 insertions(+), 1 deletion(-)
create mode 100644 sysdeps/s390/memmove-c.c
create mode 100644 sysdeps/s390/memmove.c

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index c196a19bfb..859f36f249 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -56,5 +56,6 @@ endif
ifeq ($(subdir),string)
sysdep_routines += bzero memset memset-z900 \
memcmp memcmp-z900 \
- mempcpy memcpy memcpy-z900
+ mempcpy memcpy memcpy-z900 \
+ memmove memmove-c
endif
diff --git a/sysdeps/s390/ifunc-memcpy.h b/sysdeps/s390/ifunc-memcpy.h
index 51c71baa2c..0e701968c8 100644
--- a/sysdeps/s390/ifunc-memcpy.h
+++ b/sysdeps/s390/ifunc-memcpy.h
@@ -43,6 +43,29 @@
# define HAVE_MEMCPY_Z196 HAVE_MEMCPY_IFUNC
#endif

+#if defined SHARED && defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_MEMMOVE_IFUNC 1
+#else
+# define HAVE_MEMMOVE_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_MEMMOVE_IFUNC_AND_VX_SUPPORT HAVE_MEMMOVE_IFUNC
+#else
+# define HAVE_MEMMOVE_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define MEMMOVE_DEFAULT MEMMOVE_Z13
+# define HAVE_MEMMOVE_C 0
+# define HAVE_MEMMOVE_Z13 1
+#else
+# define MEMMOVE_DEFAULT MEMMOVE_C
+# define HAVE_MEMMOVE_C 1
+# define HAVE_MEMMOVE_Z13 HAVE_MEMMOVE_IFUNC_AND_VX_SUPPORT
+#endif
+
#if HAVE_MEMCPY_Z900_G5
# define MEMCPY_Z900_G5 __memcpy_default
# define MEMPCPY_Z900_G5 __mempcpy_default
@@ -66,3 +89,15 @@
# define MEMCPY_Z196 NULL
# define MEMPCPY_Z196 NULL
#endif
+
+#if HAVE_MEMMOVE_C
+# define MEMMOVE_C __memmove_c
+#else
+# define MEMMOVE_C NULL
+#endif
+
+#if HAVE_MEMMOVE_Z13
+# define MEMMOVE_Z13 __memmove_z13
+#else
+# define MEMMOVE_Z13 NULL
+#endif
diff --git a/sysdeps/s390/memcpy-z900.S b/sysdeps/s390/memcpy-z900.S
index 3a50cf44d8..bd3b1950ee 100644
--- a/sysdeps/s390/memcpy-z900.S
+++ b/sysdeps/s390/memcpy-z900.S
@@ -182,6 +182,7 @@ ENTRY(MEMCPY_Z196)
# endif /* !defined __s390x__ */
ltgr %r4,%r4
je .L_Z196_4
+.L_Z196_start2:
aghi %r4,-1
srlg %r5,%r4,8
ltgr %r5,%r5
@@ -207,6 +208,75 @@ ENTRY(MEMCPY_Z196)
END(MEMCPY_Z196)
#endif /* HAVE_MEMCPY_Z196 */

+#if HAVE_MEMMOVE_Z13
+ENTRY(MEMMOVE_Z13)
+ .machine "z13"
+ .machinemode "zarch_nohighgprs"
+# if !defined __s390x__
+ /* Note: The 31bit dst and src pointers are prefixed with zeroes. */
+ llgfr %r4,%r4
+ llgfr %r3,%r3
+ llgfr %r2,%r2
+# endif /* !defined __s390x__ */
+ sgrk %r0,%r2,%r3
+ clgijh %r4,16,.L_MEMMOVE_Z13_LARGE
+ aghik %r5,%r4,-1
+.L_MEMMOVE_Z13_SMALL:
+ jl .L_MEMMOVE_Z13_END /* Jump away if len was zero. */
+ /* Store up to 16 bytes with vll/vstl which needs the index
+ instead of lengths. */
+ vll %v16,%r5,0(%r3)
+ vstl %v16,%r5,0(%r2)
+.L_MEMMOVE_Z13_END:
+ br %r14
+.L_MEMMOVE_Z13_LARGE:
+ lgr %r1,%r2 /* For memcpy: r1: Use as dest ;
+ r2: Return dest */
+ /* The unsigned comparison (dst - src >= len) determines if we can
+ execute the forward case with memcpy. */
+#if ! HAVE_MEMCPY_Z196
+# error The z13 variant of memmove needs the z196 variant of memcpy!
+#endif
+ clgrjhe %r0,%r4,.L_Z196_start2
+ risbgn %r5,%r4,4,128+63,60 /* r5 = r4 / 16 */
+ aghi %r4,-16
+ clgijhe %r5,8,.L_MEMMOVE_Z13_LARGE_64B
+.L_MEMMOVE_Z13_LARGE_16B_LOOP:
+ /* Store at least 16 bytes with vl/vst. The number of 16byte blocks
+ is stored in r5. */
+ vl %v16,0(%r4,%r3)
+ vst %v16,0(%r4,%r2)
+ aghi %r4,-16
+ brctg %r5,.L_MEMMOVE_Z13_LARGE_16B_LOOP
+ aghik %r5,%r4,15
+ j .L_MEMMOVE_Z13_SMALL
+.L_MEMMOVE_Z13_LARGE_64B:
+ /* Store at least 128 bytes with 4x vl/vst. The number of 64byte blocks
+ will be stored in r0. */
+ aghi %r4,-48
+ srlg %r0,%r5,2 /* r5 = %r0 / 4
+ => Number of 64byte blocks. */
+.L_MEMMOVE_Z13_LARGE_64B_LOOP:
+ vl %v20,48(%r4,%r3)
+ vl %v19,32(%r4,%r3)
+ vl %v18,16(%r4,%r3)
+ vl %v17,0(%r4,%r3)
+ vst %v20,48(%r4,%r2)
+ vst %v19,32(%r4,%r2)
+ vst %v18,16(%r4,%r2)
+ vst %v17,0(%r4,%r2)
+ aghi %r4,-64
+ brctg %r0,.L_MEMMOVE_Z13_LARGE_64B_LOOP
+ aghi %r4,48
+ /* Recalculate the number of 16byte blocks. */
+ risbg %r5,%r5,62,128+63,0 /* r5 = r5 & 3
+ => Remaining 16byte blocks. */
+ jne .L_MEMMOVE_Z13_LARGE_16B_LOOP
+ aghik %r5,%r4,15
+ j .L_MEMMOVE_Z13_SMALL
+END(MEMMOVE_Z13)
+#endif /* HAVE_MEMMOVE_Z13 */
+
#if ! HAVE_MEMCPY_IFUNC
/* If we don't use ifunc, define an alias for mem[p]cpy here.
Otherwise see sysdeps/s390/mem[p]cpy.c. */
@@ -215,10 +285,27 @@ strong_alias (MEMPCPY_DEFAULT, __mempcpy)
weak_alias (__mempcpy, mempcpy)
#endif

+#if ! HAVE_MEMMOVE_IFUNC
+/* If we don't use ifunc, define an alias for memmove here.
+ Otherwise see sysdeps/s390/memmove.c. */
+# if ! HAVE_MEMMOVE_C
+/* If the c variant is needed, then sysdeps/s390/memmove-c.c
+ defines memmove.
+ Otherwise MEMMOVE_DEFAULT is implemented here and we have to define it. */
+strong_alias (MEMMOVE_DEFAULT, memmove)
+# endif
+#endif
+
#if defined SHARED && IS_IN (libc)
/* Defines the internal symbols.
Compare to libc_hidden_[builtin_]def (mem[p]cpy) in string/mem[p]cpy.c. */
strong_alias (MEMCPY_DEFAULT, __GI_memcpy)
strong_alias (MEMPCPY_DEFAULT, __GI_mempcpy)
strong_alias (MEMPCPY_DEFAULT, __GI___mempcpy)
+# if ! HAVE_MEMMOVE_C
+/* If the c variant is needed, then sysdeps/s390/memmove-c.c
+ defines the internal symbol.
+ Otherwise MEMMOVE_DEFAULT is implemented here and we have to define it. */
+strong_alias (MEMMOVE_DEFAULT, __GI_memmove)
+# endif
#endif
diff --git a/sysdeps/s390/memmove-c.c b/sysdeps/s390/memmove-c.c
new file mode 100644
index 0000000000..be571093e0
--- /dev/null
+++ b/sysdeps/s390/memmove-c.c
@@ -0,0 +1,37 @@
+/* Fallback C version of memmove.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <ifunc-memcpy.h>
+
+#if HAVE_MEMMOVE_C
+# if HAVE_MEMMOVE_IFUNC
+/* If we use ifunc, then the memmove symbol is defined
+ in sysdeps/s390/memmove.c and we use a different name here.
+ Otherwise, we have to define memmove here or in
+ sysdeps/s390/memcpy.S depending on the used default implementation. */
+# define MEMMOVE MEMMOVE_C
+# if defined SHARED && IS_IN (libc)
+/* Define the internal symbol. */
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__memmove_c, __GI_memmove, __memmove_c);
+# endif
+# endif
+
+# include <string/memmove.c>
+#endif
diff --git a/sysdeps/s390/memmove.c b/sysdeps/s390/memmove.c
new file mode 100644
index 0000000000..ac34edf80f
--- /dev/null
+++ b/sysdeps/s390/memmove.c
@@ -0,0 +1,44 @@
+/* Multiple versions of memmove.
+ Copyright (C) 2016-2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <ifunc-memcpy.h>
+
+#if HAVE_MEMMOVE_IFUNC
+/* If we don't use ifunc, an alias is defined for memmove
+ in sysdeps/s390/memmove-c.c or sysdeps/s390/memcpy.S
+ depending on the used default implementation. */
+# undef memmove
+# define memmove __redirect_memmove
+# include <string.h>
+# include <ifunc-resolve.h>
+# undef memmove
+
+# if HAVE_MEMMOVE_C
+extern __typeof (__redirect_memmove) MEMMOVE_C attribute_hidden;
+# endif
+
+# if HAVE_MEMMOVE_Z13
+extern __typeof (__redirect_memmove) MEMMOVE_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect_memmove, memmove,
+ (HAVE_MEMMOVE_Z13 && (hwcap & HWCAP_S390_VX))
+ ? MEMMOVE_Z13
+ : MEMMOVE_DEFAULT
+ )
+#endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 6969c480cc..c05c63e006 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -126,6 +126,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_MEMCPY_IFUNC */

+#if HAVE_MEMMOVE_IFUNC
+ IFUNC_IMPL (i, name, memmove,
+# if HAVE_MEMMOVE_Z13
+ IFUNC_IMPL_ADD (array, i, memmove,
+ dl_hwcap & HWCAP_S390_VX, MEMMOVE_Z13)
+# endif
+# if HAVE_MEMMOVE_C
+ IFUNC_IMPL_ADD (array, i, memmove, 1, MEMMOVE_C)
+# endif
+ )
+#endif /* HAVE_MEMMOVE_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \

[Stefan Liebler]

The ifunc handling for strnlen is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strnlen variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strnlen variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strnlen.
* sysdeps/s390/multiarch/strnlen-c.c: Move to ...
* sysdeps/s390/strnlen-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strnlen-vx.S: Move to ...
* sysdeps/s390/strnlen-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strnlen.c: Move to ...
* sysdeps/s390/strnlen.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strnlen.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strnlen.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 +++++-
sysdeps/s390/{multiarch => }/strnlen-c.c | 18 +++++---
sysdeps/s390/{multiarch => }/strnlen-vx.S | 21 +++++++--
sysdeps/s390/{multiarch => }/strnlen.c | 23 +++++++---
7 files changed, 113 insertions(+), 21 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strnlen.h
rename sysdeps/s390/{multiarch => }/strnlen-c.c (81%)
rename sysdeps/s390/{multiarch => }/strnlen-vx.S (90%)
rename sysdeps/s390/{multiarch => }/strnlen.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index bc21b06825..4390af56b3 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -60,5 +60,6 @@ sysdep_routines += bzero memset memset-z900 \
memmove memmove-c \
strstr strstr-vx strstr-c \
memmem memmem-vx memmem-c \
- strlen strlen-vx strlen-c
+ strlen strlen-vx strlen-c \
+ strnlen strnlen-vx strnlen-c
endif
diff --git a/sysdeps/s390/ifunc-strnlen.h b/sysdeps/s390/ifunc-strnlen.h
new file mode 100644
index 0000000000..e923298887
--- /dev/null
+++ b/sysdeps/s390/ifunc-strnlen.h
@@ -0,0 +1,52 @@
+/* strnlen variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRNLEN_IFUNC 1
+#else
+# define HAVE_STRNLEN_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRNLEN_IFUNC_AND_VX_SUPPORT HAVE_STRNLEN_IFUNC
+#else
+# define HAVE_STRNLEN_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRNLEN_DEFAULT STRNLEN_Z13
+# define HAVE_STRNLEN_C 0
+# define HAVE_STRNLEN_Z13 1
+#else
+# define STRNLEN_DEFAULT STRNLEN_C
+# define HAVE_STRNLEN_C 1
+# define HAVE_STRNLEN_Z13 HAVE_STRNLEN_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRNLEN_C
+# define STRNLEN_C __strnlen_c
+#else
+# define STRNLEN_C NULL
+#endif
+
+#if HAVE_STRNLEN_Z13
+# define STRNLEN_Z13 __strnlen_vx
+#else
+# define STRNLEN_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 601523919c..35ba223c5d 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strnlen strnlen-vx strnlen-c \
- strcpy strcpy-vx \
+sysdep_routines += strcpy strcpy-vx \
stpcpy stpcpy-vx stpcpy-c \
strncpy strncpy-vx \
stpncpy stpncpy-vx stpncpy-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index c531be4bc7..680e5b738b 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -27,6 +27,7 @@
#include <ifunc-strstr.h>
#include <ifunc-memmem.h>
#include <ifunc-strlen.h>
+#include <ifunc-strnlen.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -177,6 +178,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRLEN_IFUNC */

+#if HAVE_STRNLEN_IFUNC
+ IFUNC_IMPL (i, name, strnlen,
+# if HAVE_STRNLEN_Z13
+ IFUNC_IMPL_ADD (array, i, strnlen,
+ dl_hwcap & HWCAP_S390_VX, STRNLEN_Z13)
+# endif
+# if HAVE_STRNLEN_C
+ IFUNC_IMPL_ADD (array, i, strnlen, 1, STRNLEN_C)
+# endif
+ )
+#endif /* HAVE_STRNLEN_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -187,7 +200,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcslen);

- IFUNC_VX_IMPL (strnlen);
IFUNC_VX_IMPL (wcsnlen);

IFUNC_VX_IMPL (strcpy);
diff --git a/sysdeps/s390/multiarch/strnlen-c.c b/sysdeps/s390/strnlen-c.c
similarity index 81%
rename from sysdeps/s390/multiarch/strnlen-c.c
rename to sysdeps/s390/strnlen-c.c
index 353e83ed35..c2d887f1e4 100644
--- a/sysdeps/s390/multiarch/strnlen-c.c
+++ b/sysdeps/s390/strnlen-c.c
@@ -16,15 +16,19 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRNLEN __strnlen_c
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
+#include <ifunc-strnlen.h>
+
+#if HAVE_STRNLEN_C
+# if HAVE_STRNLEN_IFUNC
+# define STRNLEN STRNLEN_C
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
__hidden_ver1 (__strnlen_c, __GI_strnlen, __strnlen_c); \
strong_alias (__strnlen_c, __strnlen_c_1); \
__hidden_ver1 (__strnlen_c_1, __GI___strnlen, __strnlen_c_1);
-# endif /* SHARED */
+# endif
+# endif

# include <string/strnlen.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strnlen-vx.S b/sysdeps/s390/strnlen-vx.S
similarity index 90%
rename from sysdeps/s390/multiarch/strnlen-vx.S
rename to sysdeps/s390/strnlen-vx.S
index fc659a956c..0b8fe3da34 100644
--- a/sysdeps/s390/multiarch/strnlen-vx.S
+++ b/sysdeps/s390/strnlen-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strnlen.h>
+
+#if HAVE_STRNLEN_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -34,7 +36,7 @@
-r5=current_len and return_value
-v16=part of s
*/
-ENTRY(__strnlen_vx)
+ENTRY(STRNLEN_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -130,5 +132,16 @@ ENTRY(__strnlen_vx)
clgrjl %r1,%r3,.Lloop64

j .Llt64
-END(__strnlen_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRNLEN_Z13)
+
+# if ! HAVE_STRNLEN_IFUNC
+strong_alias (STRNLEN_Z13, __strnlen)
+weak_alias (__strnlen, strnlen)
+# endif
+
+# if ! HAVE_STRNLEN_C && defined SHARED && IS_IN (libc)
+strong_alias (STRNLEN_Z13, __GI_strnlen)
+strong_alias (STRNLEN_Z13, __GI___strnlen)
+# endif
+
+#endif /* HAVE_STRNLEN_Z13 */
diff --git a/sysdeps/s390/multiarch/strnlen.c b/sysdeps/s390/strnlen.c
similarity index 69%
rename from sysdeps/s390/multiarch/strnlen.c
rename to sysdeps/s390/strnlen.c
index 0f9cff5d69..aa4953d503 100644
--- a/sysdeps/s390/multiarch/strnlen.c
+++ b/sysdeps/s390/strnlen.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strnlen.h>
+
+#if HAVE_STRNLEN_IFUNC
# define strnlen __redirect_strnlen
# define __strnlen __redirect___strnlen
# include <string.h>
@@ -24,9 +26,18 @@
# undef __strnlen
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc_redirected (__redirect___strnlen, __strnlen)
-weak_alias (__strnlen, strnlen)
+# if HAVE_STRNLEN_C
+extern __typeof (__redirect_strnlen) STRNLEN_C attribute_hidden;
+# endif

-#else
-# include <string/strnlen.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_STRNLEN_Z13
+extern __typeof (__redirect_strnlen) STRNLEN_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect___strnlen, __strnlen,
+ (HAVE_STRNLEN_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRNLEN_Z13
+ : STRNLEN_DEFAULT
+ )
+weak_alias (__strnlen, strnlen)
+#endif /* HAVE_STRNLEN_IFUNC */

[Stefan Liebler]

The ifunc handling for stpcpy is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove stpcpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add stpcpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for stpcpy.
* sysdeps/s390/multiarch/stpcpy-c.c: Move to ...
* sysdeps/s390/stpcpy-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/stpcpy-vx.S: Move to ...
* sysdeps/s390/stpcpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/stpcpy.c: Move to ...
* sysdeps/s390/stpcpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-stpcpy.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-stpcpy.h | 52 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 ++++++-
sysdeps/s390/{multiarch => }/stpcpy-c.c | 29 +++++++------
sysdeps/s390/{multiarch => }/stpcpy-vx.S | 20 +++++++--
sysdeps/s390/{multiarch => }/stpcpy.c | 23 ++++++++---
7 files changed, 118 insertions(+), 26 deletions(-)
create mode 100644 sysdeps/s390/ifunc-stpcpy.h
rename sysdeps/s390/{multiarch => }/stpcpy-c.c (74%)
rename sysdeps/s390/{multiarch => }/stpcpy-vx.S (90%)
rename sysdeps/s390/{multiarch => }/stpcpy.c (74%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index af9482adc3..96f1538cfd 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -62,5 +62,6 @@ sysdep_routines += bzero memset memset-z900 \
memmem memmem-vx memmem-c \
strlen strlen-vx strlen-c \
strnlen strnlen-vx strnlen-c \
- strcpy strcpy-vx strcpy-z900
+ strcpy strcpy-vx strcpy-z900 \
+ stpcpy stpcpy-vx stpcpy-c
endif
diff --git a/sysdeps/s390/ifunc-stpcpy.h b/sysdeps/s390/ifunc-stpcpy.h
new file mode 100644
index 0000000000..9a70cd7c8c
--- /dev/null
+++ b/sysdeps/s390/ifunc-stpcpy.h
@@ -0,0 +1,52 @@
+/* stpcpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STPCPY_IFUNC 1
+#else
+# define HAVE_STPCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STPCPY_IFUNC_AND_VX_SUPPORT HAVE_STPCPY_IFUNC
+#else
+# define HAVE_STPCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STPCPY_DEFAULT STPCPY_Z13
+# define HAVE_STPCPY_C 0
+# define HAVE_STPCPY_Z13 1
+#else
+# define STPCPY_DEFAULT STPCPY_C
+# define HAVE_STPCPY_C 1
+# define HAVE_STPCPY_Z13 HAVE_STPCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STPCPY_C
+# define STPCPY_C __stpcpy_c
+#else
+# define STPCPY_C NULL
+#endif
+
+#if HAVE_STPCPY_Z13
+# define STPCPY_Z13 __stpcpy_vx
+#else
+# define STPCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 50f7f0b78d..9517417dcb 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += stpcpy stpcpy-vx stpcpy-c \
- strncpy strncpy-vx \
+sysdep_routines += strncpy strncpy-vx \
stpncpy stpncpy-vx stpncpy-c \
strcat strcat-vx strcat-c \
strncat strncat-vx strncat-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 1784372db9..678ed13833 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -29,6 +29,7 @@
#include <ifunc-strlen.h>
#include <ifunc-strnlen.h>
#include <ifunc-strcpy.h>
+#include <ifunc-stpcpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -203,6 +204,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRCPY_IFUNC */

+#if HAVE_STPCPY_IFUNC
+ IFUNC_IMPL (i, name, stpcpy,
+# if HAVE_STPCPY_Z13
+ IFUNC_IMPL_ADD (array, i, stpcpy,
+ dl_hwcap & HWCAP_S390_VX, STPCPY_Z13)
+# endif
+# if HAVE_STPCPY_C
+ IFUNC_IMPL_ADD (array, i, stpcpy, 1, STPCPY_C)
+# endif
+ )
+#endif /* HAVE_STPCPY_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -217,7 +230,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcscpy);

- IFUNC_VX_IMPL (stpcpy);
IFUNC_VX_IMPL (wcpcpy);

IFUNC_VX_IMPL (strncpy);
diff --git a/sysdeps/s390/multiarch/stpcpy-c.c b/sysdeps/s390/stpcpy-c.c
similarity index 74%
rename from sysdeps/s390/multiarch/stpcpy-c.c
rename to sysdeps/s390/stpcpy-c.c
index 4a1c3e5832..76ec884627 100644
--- a/sysdeps/s390/multiarch/stpcpy-c.c
+++ b/sysdeps/s390/stpcpy-c.c
@@ -16,20 +16,25 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STPCPY __stpcpy_c
-# undef weak_alias
-# define weak_alias(a, b)
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
+#include <ifunc-stpcpy.h>
+
+#if HAVE_STPCPY_C
+# if HAVE_STPCPY_IFUNC
+# define STPCPY STPCPY_C
+
+# undef weak_alias
+# define weak_alias(a, b)
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
__hidden_ver1 (__stpcpy_c, __GI___stpcpy, __stpcpy_c);
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
strong_alias (__stpcpy_c, __stpcpy_c_1); \
__hidden_ver1 (__stpcpy_c_1, __GI_stpcpy, __stpcpy_c_1);
-# endif /* SHARED */
-
+# endif
+# endif

# include <string/stpcpy.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/stpcpy-vx.S b/sysdeps/s390/stpcpy-vx.S
similarity index 90%
rename from sysdeps/s390/multiarch/stpcpy-vx.S
rename to sysdeps/s390/stpcpy-vx.S
index 6c17def0fc..d2db02d0cd 100644
--- a/sysdeps/s390/multiarch/stpcpy-vx.S
+++ b/sysdeps/s390/stpcpy-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-stpcpy.h>
+
+#if HAVE_STPCPY_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -36,7 +38,7 @@
-v17=index of zero
-v18=part of src
*/
-ENTRY(__stpcpy_vx)
+ENTRY(STPCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -100,5 +102,15 @@ ENTRY(__stpcpy_vx)
vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
la %r2,0(%r5,%r2) /* Return pointer to zero. */
br %r14
-END(__stpcpy_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STPCPY_Z13)
+
+# if ! HAVE_STPCPY_IFUNC
+strong_alias (STPCPY_Z13, __stpcpy)
+weak_alias (__stpcpy, stpcpy)
+# endif
+
+# if ! HAVE_STPCPY_C && defined SHARED && IS_IN (libc)
+strong_alias (STPCPY_Z13, __GI_stpcpy)
+strong_alias (STPCPY_Z13, __GI___stpcpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/stpcpy.c b/sysdeps/s390/stpcpy.c
similarity index 74%
rename from sysdeps/s390/multiarch/stpcpy.c
rename to sysdeps/s390/stpcpy.c
index 654f9dfbef..670604e2de 100644
--- a/sysdeps/s390/multiarch/stpcpy.c
+++ b/sysdeps/s390/stpcpy.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-stpcpy.h>
+
+#if HAVE_STPCPY_IFUNC
# define stpcpy __redirect_stpcpy
# define __stpcpy __redirect___stpcpy
/* Omit the stpcpy inline definitions because it would redefine stpcpy. */
@@ -27,9 +29,18 @@
# undef __stpcpy
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc_redirected (__redirect___stpcpy, __stpcpy);
-weak_alias (__stpcpy, stpcpy)
+# if HAVE_STPCPY_C
+extern __typeof (__redirect_stpcpy) STPCPY_C attribute_hidden;
+# endif

-#else
-# include <string/stpcpy.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_STPCPY_Z13
+extern __typeof (__redirect_stpcpy) STPCPY_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect___stpcpy, __stpcpy,
+ (HAVE_STPCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STPCPY_Z13
+ : STPCPY_DEFAULT
+ )
+weak_alias (__stpcpy, stpcpy)
+#endif

[Stefan Liebler]

The ifunc handling for strncpy is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

Note: The fallback s390-32/s390-64 ifunc variants are now moved to
the strncpy-z900.S files. The s390-32/s390-64 files multiarch/strncpy.c
and strncpy.S are deleted.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strncpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strncpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strncpy.
* sysdeps/s390/multiarch/strncpy-vx.S: Move to ...
* sysdeps/s390/strncpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strncpy.c: Move to ...
* sysdeps/s390/strncpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strncpy.h: New file.
* sysdeps/s390/s390-64/strncpy.S: Move to ...
* sysdeps/s390/s390-64/strncpy-z900.S: ... here
and adjust ifunc handling.
* sysdeps/s390/s390-32/strncpy.S: Move to ...
* sysdeps/s390/s390-32/strncpy-z900.S: ... here
and adjust ifunc handling.
* sysdeps/s390/s390-32/multiarch/strncpy.c: Delete file.
* sysdeps/s390/s390-64/multiarch/strncpy.c: Likewise.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strncpy.h | 52 +++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 +++++-
sysdeps/s390/s390-32/multiarch/strncpy.c | 21 --------
.../s390-32/{strncpy.S => strncpy-z900.S} | 16 ++++--
sysdeps/s390/s390-64/multiarch/strncpy.c | 21 --------
.../s390-64/{strncpy.S => strncpy-z900.S} | 16 ++++--
sysdeps/s390/{multiarch => }/strncpy-vx.S | 25 ++++-----
sysdeps/s390/{multiarch => }/strncpy.c | 17 +++++-
10 files changed, 123 insertions(+), 66 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strncpy.h
delete mode 100644 sysdeps/s390/s390-32/multiarch/strncpy.c
rename sysdeps/s390/s390-32/{strncpy.S => strncpy-z900.S} (89%)
delete mode 100644 sysdeps/s390/s390-64/multiarch/strncpy.c
rename sysdeps/s390/s390-64/{strncpy.S => strncpy-z900.S} (90%)
rename sysdeps/s390/{multiarch => }/strncpy-vx.S (93%)
rename sysdeps/s390/{multiarch => }/strncpy.c (71%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 96f1538cfd..7cadf7ceba 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -63,5 +63,6 @@ sysdep_routines += bzero memset memset-z900 \
strlen strlen-vx strlen-c \
strnlen strnlen-vx strnlen-c \
strcpy strcpy-vx strcpy-z900 \
- stpcpy stpcpy-vx stpcpy-c
+ stpcpy stpcpy-vx stpcpy-c \
+ strncpy strncpy-vx strncpy-z900
endif
diff --git a/sysdeps/s390/ifunc-strncpy.h b/sysdeps/s390/ifunc-strncpy.h
new file mode 100644
index 0000000000..31e87e93c5
--- /dev/null
+++ b/sysdeps/s390/ifunc-strncpy.h
@@ -0,0 +1,52 @@
+/* strncpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRNCPY_IFUNC 1
+#else
+# define HAVE_STRNCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRNCPY_IFUNC_AND_VX_SUPPORT HAVE_STRNCPY_IFUNC
+#else
+# define HAVE_STRNCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRNCPY_DEFAULT STRNCPY_Z13
+# define HAVE_STRNCPY_Z900_G5 0
+# define HAVE_STRNCPY_Z13 1
+#else
+# define STRNCPY_DEFAULT STRNCPY_Z900_G5
+# define HAVE_STRNCPY_Z900_G5 1
+# define HAVE_STRNCPY_Z13 HAVE_STRNCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRNCPY_Z900_G5
+# define STRNCPY_Z900_G5 __strncpy_default
+#else
+# define STRNCPY_Z900_G5 NULL
+#endif
+
+#if HAVE_STRNCPY_Z13
+# define STRNCPY_Z13 __strncpy_vx
+#else
+# define STRNCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 9517417dcb..c5189b556c 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strncpy strncpy-vx \
- stpncpy stpncpy-vx stpncpy-c \
+sysdep_routines += stpncpy stpncpy-vx stpncpy-c \
strcat strcat-vx strcat-c \
strncat strncat-vx strncat-c \
strcmp strcmp-vx \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 678ed13833..d598fc5c22 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -30,6 +30,7 @@
#include <ifunc-strnlen.h>
#include <ifunc-strcpy.h>
#include <ifunc-stpcpy.h>
+#include <ifunc-strncpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -216,6 +217,19 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STPCPY_IFUNC */

+#if HAVE_STRNCPY_IFUNC
+ IFUNC_IMPL (i, name, strncpy,
+# if HAVE_STRNCPY_Z13
+ IFUNC_IMPL_ADD (array, i, strncpy,
+ dl_hwcap & HWCAP_S390_VX, STRNCPY_Z13)
+# endif
+# if HAVE_STRNCPY_Z900_G5
+ IFUNC_IMPL_ADD (array, i, strncpy, 1, STRNCPY_Z900_G5)
+# endif
+ )
+#endif /* HAVE_STRNCPY_IFUNC */
+
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -232,7 +246,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcpcpy);

- IFUNC_VX_IMPL (strncpy);
IFUNC_VX_IMPL (wcsncpy);

IFUNC_VX_IMPL (stpncpy);
diff --git a/sysdeps/s390/s390-32/multiarch/strncpy.c b/sysdeps/s390/s390-32/multiarch/strncpy.c
deleted file mode 100644
index 57f9df18d1..0000000000
--- a/sysdeps/s390/s390-32/multiarch/strncpy.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/* Multiple versions of strncpy.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* This wrapper-file is needed, because otherwise file
- sysdeps/s390/s390-[32|64]/strncpy.S will be used. */
-#include <sysdeps/s390/multiarch/strncpy.c>
diff --git a/sysdeps/s390/s390-32/strncpy.S b/sysdeps/s390/s390-32/strncpy-z900.S
similarity index 89%
rename from sysdeps/s390/s390-32/strncpy.S
rename to sysdeps/s390/s390-32/strncpy-z900.S
index 9086eb1c70..ebdaba0152 100644
--- a/sysdeps/s390/s390-32/strncpy.S
+++ b/sysdeps/s390/s390-32/strncpy-z900.S
@@ -24,10 +24,12 @@
* R4 = max of bytes to copy
*/

+#include <ifunc-strncpy.h>
#include "sysdep.h"
#include "asm-syntax.h"

-ENTRY(strncpy)
+#if HAVE_STRNCPY_Z900_G5
+ENTRY(STRNCPY_Z900_G5)
.text
st %r2,24(%r15) # save dst pointer
slr %r2,%r3 # %r3 points to src, %r2+%r3 to dst
@@ -75,5 +77,13 @@ ENTRY(strncpy)
jo .L9
.Lexit: l %r2,24(%r15) # return dst pointer
br %r14
-END(strncpy)
-libc_hidden_builtin_def (strncpy)
+END(STRNCPY_Z900_G5)
+
+# if ! HAVE_STRNCPY_IFUNC
+strong_alias (STRNCPY_Z900_G5, strncpy)
+# endif
+
+# if defined SHARED && IS_IN (libc)
+strong_alias (STRNCPY_Z900_G5, __GI_strncpy)
+# endif
+#endif
diff --git a/sysdeps/s390/s390-64/multiarch/strncpy.c b/sysdeps/s390/s390-64/multiarch/strncpy.c
deleted file mode 100644
index 57f9df18d1..0000000000
--- a/sysdeps/s390/s390-64/multiarch/strncpy.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/* Multiple versions of strncpy.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* This wrapper-file is needed, because otherwise file
- sysdeps/s390/s390-[32|64]/strncpy.S will be used. */
-#include <sysdeps/s390/multiarch/strncpy.c>
diff --git a/sysdeps/s390/s390-64/strncpy.S b/sysdeps/s390/s390-64/strncpy-z900.S
similarity index 90%
rename from sysdeps/s390/s390-64/strncpy.S
rename to sysdeps/s390/s390-64/strncpy-z900.S
index be40aa32d5..5732e6d83b 100644
--- a/sysdeps/s390/s390-64/strncpy.S
+++ b/sysdeps/s390/s390-64/strncpy-z900.S
@@ -23,10 +23,12 @@
%r3 = address of source (src)
%r4 = max of bytes to copy. */

+#include <ifunc-strncpy.h>
#include "sysdep.h"
#include "asm-syntax.h"

-ENTRY(strncpy)
+#if HAVE_STRNCPY_Z900_G5
+ENTRY(STRNCPY_Z900_G5)
.text
stg %r2,48(%r15) # save dst pointer
slgr %r2,%r3 # %r3 points to src, %r2+%r3 to dst
@@ -86,5 +88,13 @@ ENTRY(strncpy)
jo .L13
.Lexit: lg %r2,48(%r15) # return dst pointer
br %r14
-END(strncpy)
-libc_hidden_builtin_def (strncpy)
+END(STRNCPY_Z900_G5)
+
+# if ! HAVE_STRNCPY_IFUNC
+strong_alias (STRNCPY_Z900_G5, strncpy)
+# endif
+
+# if defined SHARED && IS_IN (libc)
+strong_alias (STRNCPY_Z900_G5, __GI_strncpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/strncpy-vx.S b/sysdeps/s390/strncpy-vx.S
similarity index 93%
rename from sysdeps/s390/multiarch/strncpy-vx.S
rename to sysdeps/s390/strncpy-vx.S
index 2a37b7b84e..be09ddf092 100644
--- a/sysdeps/s390/multiarch/strncpy-vx.S
+++ b/sysdeps/s390/strncpy-vx.S
@@ -16,13 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-
-# include "sysdep.h"
-# include "asm-syntax.h"
+#include <ifunc-strncpy.h>
+#include "sysdep.h"
+#include "asm-syntax.h"

.text

+#if HAVE_STRNCPY_Z13
/* char * strncpy (const char *dest, const char *src, size_t n)
Copy at most n characters of string src to dest.

@@ -40,7 +40,7 @@
-v18=part of src
-v31=register save area for r6, r7
*/
-ENTRY(__strncpy_vx)
+ENTRY(STRNCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -196,12 +196,13 @@ ENTRY(__strncpy_vx)

vl %v16,0(%r5,%r3) /* Load s. */
j .Llt64
-END(__strncpy_vx)
+END(STRNCPY_Z13)

-# define strncpy __strncpy_c
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) strong_alias(__strncpy_c, __GI_strncpy)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+# if ! HAVE_STRNCPY_IFUNC
+strong_alias (STRNCPY_Z13, strncpy)
+# endif

-/* Include strncpy-implementation in s390-32/s390-64 subdirectory. */
-#include <strncpy.S>
+# if ! HAVE_STRNCPY_Z900_G5 && defined SHARED && IS_IN (libc)
+strong_alias (STRNCPY_Z13, __GI_strncpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/strncpy.c b/sysdeps/s390/strncpy.c
similarity index 71%
rename from sysdeps/s390/multiarch/strncpy.c
rename to sysdeps/s390/strncpy.c
index 2d4c456d96..ec8a26471b 100644
--- a/sysdeps/s390/multiarch/strncpy.c
+++ b/sysdeps/s390/strncpy.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strncpy.h>
+
+#if HAVE_STRNCPY_IFUNC
# define strncpy __redirect_strncpy
/* Omit the strncpy inline definitions because it would redefine strncpy. */
# define __NO_STRING_INLINES
@@ -24,6 +26,17 @@
# undef strncpy
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strncpy, __strncpy, strncpy);
+# if HAVE_STRNCPY_Z900_G5
+extern __typeof (__redirect_strncpy) STRNCPY_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_STRNCPY_Z13
+extern __typeof (__redirect_strncpy) STRNCPY_Z13 attribute_hidden;
+# endif

+s390_libc_ifunc_expr (__redirect_strncpy, strncpy,
+ (HAVE_STRNCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRNCPY_Z13
+ : STRNCPY_DEFAULT
+ )
#endif

[Stefan Liebler]

The ifunc handling for memrchr is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove memrchr variants.
* sysdeps/s390/Makefile (sysdep_routines): Add memrchr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for memrchr.
* sysdeps/s390/multiarch/memrchr-c.c: Move to ...
* sysdeps/s390/memrchr-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/memrchr-vx.S: Move to ...
* sysdeps/s390/memrchr-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/memrchr.c: Move to ...
* sysdeps/s390/memrchr.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-memrchr.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-memrchr.h | 52 +++++++++++++++++++++++
sysdeps/s390/{multiarch => }/memrchr-c.c | 10 +++--
sysdeps/s390/{multiarch => }/memrchr-vx.S | 16 +++++--
sysdeps/s390/{multiarch => }/memrchr.c | 23 +++++++---
sysdeps/s390/multiarch/Makefile | 4 --
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
7 files changed, 102 insertions(+), 21 deletions(-)
create mode 100644 sysdeps/s390/ifunc-memrchr.h
rename sysdeps/s390/{multiarch => }/memrchr-c.c (85%)
rename sysdeps/s390/{multiarch => }/memrchr-vx.S (94%)
rename sysdeps/s390/{multiarch => }/memrchr.c (68%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 649f29adc2..0e86242c5d 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -78,5 +78,6 @@ sysdep_routines += bzero memset memset-z900 \
strcspn strcspn-vx strcspn-c \
memchr memchr-vx memchr-z900 \
rawmemchr rawmemchr-vx rawmemchr-c \
- memccpy memccpy-vx memccpy-c
+ memccpy memccpy-vx memccpy-c \
+ memrchr memrchr-vx memrchr-c
endif
diff --git a/sysdeps/s390/ifunc-memrchr.h b/sysdeps/s390/ifunc-memrchr.h
new file mode 100644
index 0000000000..9d80d5528d
--- /dev/null
+++ b/sysdeps/s390/ifunc-memrchr.h
@@ -0,0 +1,52 @@
+/* memrchr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_MEMRCHR_IFUNC 1
+#else
+# define HAVE_MEMRCHR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_MEMRCHR_IFUNC_AND_VX_SUPPORT HAVE_MEMRCHR_IFUNC
+#else
+# define HAVE_MEMRCHR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define MEMRCHR_DEFAULT MEMRCHR_Z13
+# define HAVE_MEMRCHR_C 0
+# define HAVE_MEMRCHR_Z13 1
+#else
+# define MEMRCHR_DEFAULT MEMRCHR_C
+# define HAVE_MEMRCHR_C 1
+# define HAVE_MEMRCHR_Z13 HAVE_MEMRCHR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_MEMRCHR_C
+# define MEMRCHR_C __memrchr_c
+#else
+# define MEMRCHR_C NULL
+#endif
+
+#if HAVE_MEMRCHR_Z13
+# define MEMRCHR_Z13 __memrchr_vx
+#else
+# define MEMRCHR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/memrchr-c.c b/sysdeps/s390/memrchr-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/memrchr-c.c
rename to sysdeps/s390/memrchr-c.c
index 1e3c914a5d..333c0fc8d1 100644
--- a/sysdeps/s390/multiarch/memrchr-c.c
+++ b/sysdeps/s390/memrchr-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define MEMRCHR __memrchr_c
+#include <ifunc-memrchr.h>
+
+#if HAVE_MEMRCHR_C
+# if HAVE_MEMRCHR_IFUNC
+# define MEMRCHR MEMRCHR_C
+# endif

-# include <string.h>
-extern __typeof (__memrchr) __memrchr_c;
# include <string/memrchr.c>
#endif
diff --git a/sysdeps/s390/multiarch/memrchr-vx.S b/sysdeps/s390/memrchr-vx.S
similarity index 94%
rename from sysdeps/s390/multiarch/memrchr-vx.S
rename to sysdeps/s390/memrchr-vx.S
index 8e81f5ed75..ba832f1b21 100644
--- a/sysdeps/s390/multiarch/memrchr-vx.S
+++ b/sysdeps/s390/memrchr-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-memrchr.h>
+
+#if HAVE_MEMRCHR_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -40,7 +42,7 @@
-v18=c replicated
-v20=permute pattern
*/
-ENTRY(__memrchr_vx)
+ENTRY(MEMRCHR_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -156,5 +158,11 @@ ENTRY(__memrchr_vx)

clgijhe %r4,64,.Lloop64 /* If n >= 64 -> loop64. */
j .Llt64
-END(__memrchr_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(MEMRCHR_Z13)
+
+# if ! HAVE_MEMRCHR_IFUNC
+strong_alias (MEMRCHR_Z13, __memrchr)
+weak_alias (__memrchr, memrchr)
+# endif
+
+#endif /* HAVE_MEMRCHR_Z13 */
diff --git a/sysdeps/s390/multiarch/memrchr.c b/sysdeps/s390/memrchr.c
similarity index 68%
rename from sysdeps/s390/multiarch/memrchr.c
rename to sysdeps/s390/memrchr.c
index 43a44abcf6..d995e9961c 100644
--- a/sysdeps/s390/multiarch/memrchr.c
+++ b/sysdeps/s390/memrchr.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-memrchr.h>
+
+#if HAVE_MEMRCHR_IFUNC
# include <string.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__memrchr)
-weak_alias (__memrchr, memrchr)
+# if HAVE_MEMRCHR_C
+extern __typeof (__memrchr) MEMRCHR_C attribute_hidden;
+# endif

-#else
-# include <string/memrchr.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_MEMRCHR_Z13
+extern __typeof (__memrchr) MEMRCHR_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__memrchr, __memrchr,
+ (HAVE_MEMRCHR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? MEMRCHR_Z13
+ : MEMRCHR_DEFAULT
+ )
+weak_alias (__memrchr, memrchr)
+#endif /* HAVE_MEMRCHR_IFUNC */
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index d5a32fc309..260b514936 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,7 +1,3 @@
-ifeq ($(subdir),string)
-sysdep_routines += memrchr memrchr-vx memrchr-c
-endif
-
ifeq ($(subdir),wcsmbs)
sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsnlen wcsnlen-vx wcsnlen-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index b8917747f0..0f01b99691 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -45,6 +45,7 @@
#include <ifunc-memchr.h>
#include <ifunc-rawmemchr.h>
#include <ifunc-memccpy.h>
+#include <ifunc-memrchr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -411,6 +412,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_MEMCCPY_IFUNC */

+#if HAVE_MEMRCHR_IFUNC
+ IFUNC_IMPL (i, name, memrchr,
+# if HAVE_MEMRCHR_Z13
+ IFUNC_IMPL_ADD (array, i, memrchr,
+ dl_hwcap & HWCAP_S390_VX, MEMRCHR_Z13)
+# endif
+# if HAVE_MEMRCHR_C
+ IFUNC_IMPL_ADD (array, i, memrchr, 1, MEMRCHR_C)
+# endif
+ )
+#endif /* HAVE_MEMRCHR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -457,8 +470,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wmemcmp);

- IFUNC_VX_IMPL (memrchr);
-
#endif /* HAVE_S390_VX_ASM_SUPPORT */

return i;

[Stefan Liebler]

The ifunc handling for wcpcpy is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcpcpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcpcpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcpcpy.
* sysdeps/s390/multiarch/wcpcpy-c.c: Move to ...
* sysdeps/s390/wcpcpy-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcpcpy-vx.S: Move to ...
* sysdeps/s390/wcpcpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcpcpy.c: Move to ...
* sysdeps/s390/wcpcpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcpcpy.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcpcpy.h | 53 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcpcpy-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcpcpy-vx.S | 16 ++++---
sysdeps/s390/{multiarch => }/wcpcpy.c | 23 +++++++---
7 files changed, 103 insertions(+), 20 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcpcpy.h
rename sysdeps/s390/{multiarch => }/wcpcpy-c.c (86%)
rename sysdeps/s390/{multiarch => }/wcpcpy-vx.S (94%)
rename sysdeps/s390/{multiarch => }/wcpcpy.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index f2fc88af26..0fc0cdaee9 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -85,5 +85,6 @@ endif
ifeq ($(subdir),wcsmbs)
sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsnlen wcsnlen-vx wcsnlen-c \
- wcscpy wcscpy-vx wcscpy-c
+ wcscpy wcscpy-vx wcscpy-c \
+ wcpcpy wcpcpy-vx wcpcpy-c
endif
diff --git a/sysdeps/s390/ifunc-wcpcpy.h b/sysdeps/s390/ifunc-wcpcpy.h
new file mode 100644
index 0000000000..0d5e2ba1a0
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcpcpy.h
@@ -0,0 +1,53 @@
+/* wcpcpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCPCPY_IFUNC 1
+#else
+# define HAVE_WCPCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCPCPY_IFUNC_AND_VX_SUPPORT HAVE_WCPCPY_IFUNC
+#else
+# define HAVE_WCPCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCPCPY_DEFAULT WCPCPY_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCPCPY_C 1
+# define HAVE_WCPCPY_Z13 1
+#else
+# define WCPCPY_DEFAULT WCPCPY_C
+# define HAVE_WCPCPY_C 1
+# define HAVE_WCPCPY_Z13 HAVE_WCPCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCPCPY_C
+# define WCPCPY_C __wcpcpy_c
+#else
+# define WCPCPY_C NULL
+#endif
+
+#if HAVE_WCPCPY_Z13
+# define WCPCPY_Z13 __wcpcpy_vx
+#else
+# define WCPCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 8828897a59..7d7b05dcf2 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcpcpy wcpcpy-vx wcpcpy-c \
- wcsncpy wcsncpy-vx wcsncpy-c \
+sysdep_routines += wcsncpy wcsncpy-vx wcsncpy-c \
wcpncpy wcpncpy-vx wcpncpy-c \
wcscat wcscat-vx wcscat-c \
wcsncat wcsncat-vx wcsncat-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index aac8f4ea46..656ab59db6 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -49,6 +49,7 @@
#include <ifunc-wcslen.h>
#include <ifunc-wcsnlen.h>
#include <ifunc-wcscpy.h>
+#include <ifunc-wcpcpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -463,6 +464,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSCPY_IFUNC */

+#if HAVE_WCPCPY_IFUNC
+ IFUNC_IMPL (i, name, wcpcpy,
+# if HAVE_WCPCPY_Z13
+ IFUNC_IMPL_ADD (array, i, wcpcpy,
+ dl_hwcap & HWCAP_S390_VX, WCPCPY_Z13)
+# endif
+# if HAVE_WCPCPY_C
+ IFUNC_IMPL_ADD (array, i, wcpcpy, 1, WCPCPY_C)
+# endif
+ )
+#endif /* HAVE_WCPCPY_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -471,8 +484,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcpcpy);
-
IFUNC_VX_IMPL (wcsncpy);

IFUNC_VX_IMPL (wcpncpy);
diff --git a/sysdeps/s390/multiarch/wcpcpy-c.c b/sysdeps/s390/wcpcpy-c.c
similarity index 86%
rename from sysdeps/s390/multiarch/wcpcpy-c.c
rename to sysdeps/s390/wcpcpy-c.c
index e3282fde19..ed1539cde2 100644
--- a/sysdeps/s390/multiarch/wcpcpy-c.c
+++ b/sysdeps/s390/wcpcpy-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCPCPY __wcpcpy_c
+#include <ifunc-wcpcpy.h>
+
+#if HAVE_WCPCPY_C
+# if HAVE_WCPCPY_IFUNC || HAVE_WCPCPY_Z13
+# define WCPCPY WCPCPY_C
+# endif

-# include <wchar.h>
-extern __typeof (__wcpcpy) __wcpcpy_c;
# include <wcsmbs/wcpcpy.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcpcpy-vx.S b/sysdeps/s390/wcpcpy-vx.S
similarity index 94%
rename from sysdeps/s390/multiarch/wcpcpy-vx.S
rename to sysdeps/s390/wcpcpy-vx.S
index bff6e85628..5154ad4461 100644
--- a/sysdeps/s390/multiarch/wcpcpy-vx.S
+++ b/sysdeps/s390/wcpcpy-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcpcpy.h>
+#if HAVE_WCPCPY_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -37,7 +38,7 @@
-v17=index of zero
-v18=part of src
*/
-ENTRY(__wcpcpy_vx)
+ENTRY(WCPCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -109,6 +110,11 @@ ENTRY(__wcpcpy_vx)
br %r14

.Lfallback:
- jg __wcpcpy_c
-END(__wcpcpy_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCPCPY_C
+END(WCPCPY_Z13)
+
+# if ! HAVE_WCPCPY_IFUNC
+strong_alias (WCPCPY_Z13, __wcpcpy)
+weak_alias (__wcpcpy, wcpcpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcpcpy.c b/sysdeps/s390/wcpcpy.c
similarity index 70%
rename from sysdeps/s390/multiarch/wcpcpy.c
rename to sysdeps/s390/wcpcpy.c
index f19d376d85..34ac68b31f 100644
--- a/sysdeps/s390/multiarch/wcpcpy.c
+++ b/sysdeps/s390/wcpcpy.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcpcpy.h>
+
+#if HAVE_WCPCPY_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__wcpcpy)
-weak_alias (__wcpcpy, wcpcpy)
+# if HAVE_WCPCPY_C
+extern __typeof (__wcpcpy) WCPCPY_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcpcpy.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCPCPY_Z13
+extern __typeof (__wcpcpy) WCPCPY_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__wcpcpy, __wcpcpy,
+ (HAVE_WCPCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCPCPY_Z13
+ : WCPCPY_DEFAULT
+ )
+weak_alias (__wcpcpy, wcpcpy)
+#endif

[Stefan Liebler]

The ifunc handling for wcsncmp is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcsncmp variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcsncmp variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcsncmp.
* sysdeps/s390/multiarch/wcsncmp-c.c: Move to ...
* sysdeps/s390/wcsncmp-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsncmp-vx.S: Move to ...
* sysdeps/s390/wcsncmp-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsncmp.c: Move to ...
* sysdeps/s390/wcsncmp.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcsncmp.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcsncmp.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcsncmp-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcsncmp-vx.S | 13 ++++--
sysdeps/s390/{multiarch => }/wcsncmp.c | 21 ++++++---
7 files changed, 99 insertions(+), 18 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcsncmp.h
rename sysdeps/s390/{multiarch => }/wcsncmp-c.c (85%)
rename sysdeps/s390/{multiarch => }/wcsncmp-vx.S (97%)
rename sysdeps/s390/{multiarch => }/wcsncmp.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 00398ba6c2..e1253c04be 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -91,5 +91,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcpncpy wcpncpy-vx wcpncpy-c \
wcscat wcscat-vx wcscat-c \
wcsncat wcsncat-vx wcsncat-c \
- wcscmp wcscmp-vx wcscmp-c
+ wcscmp wcscmp-vx wcscmp-c \
+ wcsncmp wcsncmp-vx wcsncmp-c
endif
diff --git a/sysdeps/s390/ifunc-wcsncmp.h b/sysdeps/s390/ifunc-wcsncmp.h
new file mode 100644
index 0000000000..3a22bc9c12
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcsncmp.h
@@ -0,0 +1,52 @@
+/* wcsncmp variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSNCMP_IFUNC 1
+#else
+# define HAVE_WCSNCMP_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSNCMP_IFUNC_AND_VX_SUPPORT HAVE_WCSNCMP_IFUNC
+#else
+# define HAVE_WCSNCMP_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSNCMP_DEFAULT WCSNCMP_Z13
+# define HAVE_WCSNCMP_C 0
+# define HAVE_WCSNCMP_Z13 1
+#else
+# define WCSNCMP_DEFAULT WCSNCMP_C
+# define HAVE_WCSNCMP_C 1
+# define HAVE_WCSNCMP_Z13 HAVE_WCSNCMP_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSNCMP_C
+# define WCSNCMP_C __wcsncmp_c
+#else
+# define WCSNCMP_C NULL
+#endif
+
+#if HAVE_WCSNCMP_Z13
+# define WCSNCMP_Z13 __wcsncmp_vx
+#else
+# define WCSNCMP_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 70162d0eaa..d1740fa8e4 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcsncmp wcsncmp-vx wcsncmp-c \
- wcschr wcschr-vx wcschr-c \
+sysdep_routines += wcschr wcschr-vx wcschr-c \
wcschrnul wcschrnul-vx wcschrnul-c \
wcsrchr wcsrchr-vx wcsrchr-c \
wcsspn wcsspn-vx wcsspn-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index f461063c80..8c9e788ae5 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -55,6 +55,7 @@
#include <ifunc-wcscat.h>
#include <ifunc-wcsncat.h>
#include <ifunc-wcscmp.h>
+#include <ifunc-wcsncmp.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -541,6 +542,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSCMP_IFUNC */

+#if HAVE_WCSNCMP_IFUNC
+ IFUNC_IMPL (i, name, wcsncmp,
+# if HAVE_WCSNCMP_Z13
+ IFUNC_IMPL_ADD (array, i, wcsncmp,
+ dl_hwcap & HWCAP_S390_VX, WCSNCMP_Z13)
+# endif
+# if HAVE_WCSNCMP_C
+ IFUNC_IMPL_ADD (array, i, wcsncmp, 1, WCSNCMP_C)
+# endif
+ )
+#endif /* HAVE_WCSNCMP_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -549,8 +562,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcsncmp);
-
IFUNC_VX_IMPL (wcschr);

IFUNC_VX_IMPL (wcschrnul);
diff --git a/sysdeps/s390/multiarch/wcsncmp-c.c b/sysdeps/s390/wcsncmp-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/wcsncmp-c.c
rename to sysdeps/s390/wcsncmp-c.c
index 92ab5e8b50..a9ccb8de82 100644
--- a/sysdeps/s390/multiarch/wcsncmp-c.c
+++ b/sysdeps/s390/wcsncmp-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSNCMP __wcsncmp_c
+#include <ifunc-wcsncmp.h>
+
+#if HAVE_WCSNCMP_C
+# if HAVE_WCSNCMP_IFUNC
+# define WCSNCMP WCSNCMP_C
+# endif

-# include <wchar.h>
-extern __typeof (wcsncmp) __wcsncmp_c;
# include <wcsmbs/wcsncmp.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcsncmp-vx.S b/sysdeps/s390/wcsncmp-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/wcsncmp-vx.S
rename to sysdeps/s390/wcsncmp-vx.S
index 34c203bc90..9bdd21acef 100644
--- a/sysdeps/s390/multiarch/wcsncmp-vx.S
+++ b/sysdeps/s390/wcsncmp-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsncmp.h>
+#if HAVE_WCSNCMP_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -37,7 +38,7 @@
-v17=part of s2
-v18=index of unequal
*/
-ENTRY(__wcsncmp_vx)
+ENTRY(WCSNCMP_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -173,5 +174,9 @@ ENTRY(__wcsncmp_vx)
lghi %r1,-1
locgrl %r2,%r1
br %r14
-END(__wcsncmp_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(WCSNCMP_Z13)
+
+# if ! HAVE_WCSNCMP_IFUNC
+strong_alias (WCSNCMP_Z13, wcsncmp)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcsncmp.c b/sysdeps/s390/wcsncmp.c
similarity index 69%
rename from sysdeps/s390/multiarch/wcsncmp.c
rename to sysdeps/s390/wcsncmp.c
index ee5f08c1e4..101db44591 100644
--- a/sysdeps/s390/multiarch/wcsncmp.c
+++ b/sysdeps/s390/wcsncmp.c
@@ -16,12 +16,23 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsncmp.h>
+
+#if HAVE_WCSNCMP_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2 (__wcsncmp, wcsncmp)
+# if HAVE_WCSNCMP_C
+extern __typeof (wcsncmp) WCSNCMP_C attribute_hidden;
+# endif
+
+# if HAVE_WCSNCMP_Z13
+extern __typeof (wcsncmp) WCSNCMP_Z13 attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcsncmp.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (wcsncmp, wcsncmp,
+ (HAVE_WCSNCMP_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSNCMP_Z13
+ : WCSNCMP_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for wcscpy is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcscpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcscpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcscpy.
* sysdeps/s390/multiarch/wcscpy-c.c: Move to ...
* sysdeps/s390/wcscpy-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcscpy-vx.S: Move to ...
* sysdeps/s390/wcscpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcscpy.c: Move to ...
* sysdeps/s390/wcscpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcscpy.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcscpy.h | 53 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcscpy-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcscpy-vx.S | 15 ++++---
sysdeps/s390/{multiarch => }/wcscpy.c | 21 +++++++---
7 files changed, 101 insertions(+), 19 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcscpy.h
rename sysdeps/s390/{multiarch => }/wcscpy-c.c (86%)
rename sysdeps/s390/{multiarch => }/wcscpy-vx.S (95%)
rename sysdeps/s390/{multiarch => }/wcscpy.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 590d2207b7..f2fc88af26 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -84,5 +84,6 @@ endif

ifeq ($(subdir),wcsmbs)
sysdep_routines += wcslen wcslen-vx wcslen-c \
- wcsnlen wcsnlen-vx wcsnlen-c
+ wcsnlen wcsnlen-vx wcsnlen-c \
+ wcscpy wcscpy-vx wcscpy-c
endif
diff --git a/sysdeps/s390/ifunc-wcscpy.h b/sysdeps/s390/ifunc-wcscpy.h
new file mode 100644
index 0000000000..fba7c9c7a7
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcscpy.h
@@ -0,0 +1,53 @@
+/* wcscpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSCPY_IFUNC 1
+#else
+# define HAVE_WCSCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSCPY_IFUNC_AND_VX_SUPPORT HAVE_WCSCPY_IFUNC
+#else
+# define HAVE_WCSCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSCPY_DEFAULT WCSCPY_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSCPY_C 1
+# define HAVE_WCSCPY_Z13 1
+#else
+# define WCSCPY_DEFAULT WCSCPY_C
+# define HAVE_WCSCPY_C 1
+# define HAVE_WCSCPY_Z13 HAVE_WCSCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSCPY_C
+# define WCSCPY_C __wcscpy_c
+#else
+# define WCSCPY_C NULL
+#endif
+
+#if HAVE_WCSCPY_Z13
+# define WCSCPY_Z13 __wcscpy_vx
+#else
+# define WCSCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index ce2e7ce5f4..8828897a59 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcscpy wcscpy-vx wcscpy-c \
- wcpcpy wcpcpy-vx wcpcpy-c \
+sysdep_routines += wcpcpy wcpcpy-vx wcpcpy-c \
wcsncpy wcsncpy-vx wcsncpy-c \
wcpncpy wcpncpy-vx wcpncpy-c \
wcscat wcscat-vx wcscat-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index c199fd0e0b..aac8f4ea46 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -48,6 +48,7 @@
#include <ifunc-memrchr.h>
#include <ifunc-wcslen.h>
#include <ifunc-wcsnlen.h>
+#include <ifunc-wcscpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -450,6 +451,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSNLEN_IFUNC */

+#if HAVE_WCSCPY_IFUNC
+ IFUNC_IMPL (i, name, wcscpy,
+# if HAVE_WCSCPY_Z13
+ IFUNC_IMPL_ADD (array, i, wcscpy,
+ dl_hwcap & HWCAP_S390_VX, WCSCPY_Z13)
+# endif
+# if HAVE_WCSCPY_C
+ IFUNC_IMPL_ADD (array, i, wcscpy, 1, WCSCPY_C)
+# endif
+ )
+#endif /* HAVE_WCSCPY_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -458,8 +471,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcscpy);
-
IFUNC_VX_IMPL (wcpcpy);

IFUNC_VX_IMPL (wcsncpy);
diff --git a/sysdeps/s390/multiarch/wcscpy-c.c b/sysdeps/s390/wcscpy-c.c
similarity index 86%
rename from sysdeps/s390/multiarch/wcscpy-c.c
rename to sysdeps/s390/wcscpy-c.c
index 4a510f466b..db2967f47d 100644
--- a/sysdeps/s390/multiarch/wcscpy-c.c
+++ b/sysdeps/s390/wcscpy-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSCPY __wcscpy_c
+#include <ifunc-wcscpy.h>
+
+#if HAVE_WCSCPY_C
+# if HAVE_WCSCPY_IFUNC || HAVE_WCSCPY_Z13
+# define WCSCPY WCSCPY_C
+# endif

-# include <wchar.h>
-extern __typeof (wcscpy) __wcscpy_c;
# include <wcsmbs/wcscpy.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcscpy-vx.S b/sysdeps/s390/wcscpy-vx.S
similarity index 95%
rename from sysdeps/s390/multiarch/wcscpy-vx.S
rename to sysdeps/s390/wcscpy-vx.S
index c2e81055be..8fe12f4d8b 100644
--- a/sysdeps/s390/multiarch/wcscpy-vx.S
+++ b/sysdeps/s390/wcscpy-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcscpy.h>
+#if HAVE_WCSCPY_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -37,7 +38,7 @@
-v17=index of zero
-v18=part of src
*/
-ENTRY(__wcscpy_vx)
+ENTRY(WCSCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -106,6 +107,10 @@ ENTRY(__wcscpy_vx)
br %r14

.Lfallback:
- jg __wcscpy_c
-END(__wcscpy_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSCPY_C
+END(WCSCPY_Z13)
+
+# if ! HAVE_WCSCPY_IFUNC
+strong_alias (WCSCPY_Z13, wcscpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcscpy.c b/sysdeps/s390/wcscpy.c
similarity index 70%
rename from sysdeps/s390/multiarch/wcscpy.c
rename to sysdeps/s390/wcscpy.c
index e69baa6c59..51f07327da 100644
--- a/sysdeps/s390/multiarch/wcscpy.c
+++ b/sysdeps/s390/wcscpy.c
@@ -16,12 +16,23 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcscpy.h>
+
+#if HAVE_WCSCPY_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2 (__wcscpy, wcscpy)
+# if HAVE_WCSCPY_C
+extern __typeof (wcscpy) WCSCPY_C attribute_hidden;
+# endif
+
+# if HAVE_WCSCPY_Z13
+extern __typeof (wcscpy) WCSCPY_Z13 attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcscpy.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (wcscpy, wcscpy,
+ (HAVE_WCSCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSCPY_Z13
+ : WCSCPY_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for memchr is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

Note: The fallback s390-32/s390-64 ifunc variants with srst instruction
are now moved to the unified memchr-z900.S file which can be used for
31/64bit. The s390-32/s390-64 files multiarch/memchr.c and memchr.S
are deleted.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove memchr variants.
* sysdeps/s390/Makefile (sysdep_routines): Add memchr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for memchr.
* sysdeps/s390/multiarch/memchr-vx.S: Move to ...
* sysdeps/s390/memchr-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/memchr.c: Move to ...
* sysdeps/s390/memchr.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-memchr.h: New file.
* sysdeps/s390/s390-64/memchr.S: Move to ...
* sysdeps/s390/memchr-z900.S: ... here and adjust to be usable
for 31/64bit and ifunc handling.
* sysdeps/s390/s390-32/multiarch/memchr.c: Delete file.
* sysdeps/s390/s390-64/multiarch/memchr.c: Likewise.
* sysdeps/s390/s390-32/memchr.S: Likewise.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-memchr.h | 52 +++++++++++++++++++
sysdeps/s390/{multiarch => }/memchr-vx.S | 20 ++++---
.../s390/{s390-64/memchr.S => memchr-z900.S} | 39 +++++++++++---
sysdeps/s390/{multiarch => }/memchr.c | 18 ++++++-
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 ++++-
sysdeps/s390/s390-32/memchr.S | 41 ---------------
sysdeps/s390/s390-32/multiarch/memchr.c | 21 --------
sysdeps/s390/s390-64/multiarch/memchr.c | 21 --------
10 files changed, 127 insertions(+), 105 deletions(-)
create mode 100644 sysdeps/s390/ifunc-memchr.h
rename sysdeps/s390/{multiarch => }/memchr-vx.S (92%)
rename sysdeps/s390/{s390-64/memchr.S => memchr-z900.S} (63%)
rename sysdeps/s390/{multiarch => }/memchr.c (68%)
delete mode 100644 sysdeps/s390/s390-32/memchr.S
delete mode 100644 sysdeps/s390/s390-32/multiarch/memchr.c
delete mode 100644 sysdeps/s390/s390-64/multiarch/memchr.c

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index c163969652..8f11ed5463 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -75,5 +75,6 @@ sysdep_routines += bzero memset memset-z900 \
strrchr strrchr-vx strrchr-c \
strspn strspn-vx strspn-c \
strpbrk strpbrk-vx strpbrk-c \
- strcspn strcspn-vx strcspn-c
+ strcspn strcspn-vx strcspn-c \
+ memchr memchr-vx memchr-z900
endif
diff --git a/sysdeps/s390/ifunc-memchr.h b/sysdeps/s390/ifunc-memchr.h
new file mode 100644
index 0000000000..5d1327b453
--- /dev/null
+++ b/sysdeps/s390/ifunc-memchr.h
@@ -0,0 +1,52 @@
+/* memchr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_MEMCHR_IFUNC 1
+#else
+# define HAVE_MEMCHR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_MEMCHR_IFUNC_AND_VX_SUPPORT HAVE_MEMCHR_IFUNC
+#else
+# define HAVE_MEMCHR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define MEMCHR_DEFAULT MEMCHR_Z13
+# define HAVE_MEMCHR_Z900_G5 0
+# define HAVE_MEMCHR_Z13 1
+#else
+# define MEMCHR_DEFAULT MEMCHR_Z900_G5
+# define HAVE_MEMCHR_Z900_G5 1
+# define HAVE_MEMCHR_Z13 HAVE_MEMCHR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_MEMCHR_Z900_G5
+# define MEMCHR_Z900_G5 __memchr_default
+#else
+# define MEMCHR_Z900_G5 NULL
+#endif
+
+#if HAVE_MEMCHR_Z13
+# define MEMCHR_Z13 __memchr_vx
+#else
+# define MEMCHR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/memchr-vx.S b/sysdeps/s390/memchr-vx.S
similarity index 92%
rename from sysdeps/s390/multiarch/memchr-vx.S
rename to sysdeps/s390/memchr-vx.S
index 77d31e0036..274e7971ca 100644
--- a/sysdeps/s390/multiarch/memchr-vx.S
+++ b/sysdeps/s390/memchr-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-memchr.h>
+#if HAVE_MEMCHR_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -38,7 +39,7 @@
-v17=index of found c
-v18=c replicated
*/
-ENTRY(__memchr_vx)
+ENTRY(MEMCHR_Z13)

.machine "z13"
.machinemode "zarch_nohighgprs"
@@ -149,11 +150,14 @@ ENTRY(__memchr_vx)
clgrjl %r0,%r4,.Lloop64

j .Llt64
-END(__memchr_vx)
+END(MEMCHR_Z13)

-# define memchr __memchr_c
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) strong_alias(__memchr_c, __GI_memchr)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+# if ! HAVE_MEMCHR_IFUNC
+strong_alias (MEMCHR_Z13, __memchr)
+weak_alias (__memchr, memchr)
+# endif

-#include <memchr.S>
+# if ! HAVE_MEMCHR_Z900_G5 && defined SHARED && IS_IN (libc)
+strong_alias (MEMCHR_Z13, __GI_memchr)
+# endif
+#endif
diff --git a/sysdeps/s390/s390-64/memchr.S b/sysdeps/s390/memchr-z900.S
similarity index 63%
rename from sysdeps/s390/s390-64/memchr.S
rename to sysdeps/s390/memchr-z900.S
index a19fcafa14..c016bc41c6 100644
--- a/sysdeps/s390/s390-64/memchr.S
+++ b/sysdeps/s390/memchr-z900.S
@@ -1,4 +1,4 @@
-/* Search a character in a block of memory. 64 bit S/390 version.
+/* Search a character in a block of memory. 31/64 bit S/390 version.
Copyright (C) 2001-2018 Free Software Foundation, Inc.
Contributed by Martin Schwidefsky (***@de.ibm.com).
This file is part of the GNU C Library.
@@ -22,19 +22,42 @@
%r3 = character to find
%r4 = number of bytes to search. */

+#include <ifunc-memchr.h>
#include "sysdep.h"
#include "asm-syntax.h"

+#if HAVE_MEMCHR_Z900_G5
+# if defined __s390x__
+# define SLGR slgr
+# define LGHI lghi
+# define NGR ngr
+# define LGR lgr
+# else
+# define SLGR slr
+# define LGHI lhi
+# define NGR nr
+# define LGR lr
+# endif /* ! defined __s390x__ */
+
.text
-ENTRY(memchr)
- lghi %r0,0xff
- ngr %r0,%r3
- lgr %r1,%r2
+ENTRY(MEMCHR_Z900_G5)
+ LGHI %r0,0xff
+ NGR %r0,%r3
+ LGR %r1,%r2
la %r2,0(%r4,%r1)
0: srst %r2,%r1
jo 0b
brc 13,1f
- slgr %r2,%r2
+ SLGR %r2,%r2
1: br %r14
-END(memchr)
-libc_hidden_builtin_def (memchr)
+END(MEMCHR_Z900_G5)
+
+# if ! HAVE_MEMCHR_IFUNC
+strong_alias (MEMCHR_Z900_G5, __memchr)
+weak_alias (__memchr, memchr)
+# endif
+
+# if defined SHARED && IS_IN (libc)
+strong_alias (MEMCHR_Z900_G5, __GI_memchr)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/memchr.c b/sysdeps/s390/memchr.c
similarity index 68%
rename from sysdeps/s390/multiarch/memchr.c
rename to sysdeps/s390/memchr.c
index 3885ebaa4d..490f1b6600 100644
--- a/sysdeps/s390/multiarch/memchr.c
+++ b/sysdeps/s390/memchr.c
@@ -16,12 +16,26 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-memchr.h>
+
+#if HAVE_MEMCHR_IFUNC
# define memchr __redirect_memchr
# include <string.h>
# undef memchr
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_memchr, __memchr, memchr)
+# if HAVE_MEMCHR_Z900_G5
+extern __typeof (__redirect_memchr) MEMCHR_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_MEMCHR_Z13
+extern __typeof (__redirect_memchr) MEMCHR_Z13 attribute_hidden;
+# endif

+s390_libc_ifunc_expr (__redirect_memchr, __memchr,
+ (HAVE_MEMCHR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? MEMCHR_Z13
+ : MEMCHR_DEFAULT
+ )
+weak_alias (__memchr, memchr)
#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 1578f21af4..fa1f7b81db 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += memchr memchr-vx \
- rawmemchr rawmemchr-vx rawmemchr-c \
+sysdep_routines += rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 2d48c99c8d..b4be014042 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -42,6 +42,7 @@
#include <ifunc-strspn.h>
#include <ifunc-strpbrk.h>
#include <ifunc-strcspn.h>
+#include <ifunc-memchr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -372,6 +373,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRCSPN_IFUNC */

+#if HAVE_MEMCHR_IFUNC
+ IFUNC_IMPL (i, name, memchr,
+# if HAVE_MEMCHR_Z13
+ IFUNC_IMPL_ADD (array, i, memchr,
+ dl_hwcap & HWCAP_S390_VX, MEMCHR_Z13)
+# endif
+# if HAVE_MEMCHR_Z900_G5
+ IFUNC_IMPL_ADD (array, i, memchr, 1, MEMCHR_Z900_G5)
+# endif
+ )
+#endif /* HAVE_MEMCHR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -412,7 +425,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcscspn);

- IFUNC_VX_IMPL (memchr);
IFUNC_VX_IMPL (wmemchr);
IFUNC_VX_IMPL (rawmemchr);

diff --git a/sysdeps/s390/s390-32/memchr.S b/sysdeps/s390/s390-32/memchr.S
deleted file mode 100644
index 54f9b85f57..0000000000
--- a/sysdeps/s390/s390-32/memchr.S
+++ /dev/null
@@ -1,41 +0,0 @@
-/* Search a character in a block of memory. For IBM S390
- Copyright (C) 2000-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
- Contributed by Martin Schwidefsky (***@de.ibm.com).
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/*
- * R2 = address to memory area
- * R3 = character to find
- * R4 = number of bytes to search
- */
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
- .text
-ENTRY(memchr)
- lhi %r0,0xff
- nr %r0,%r3
- lr %r1,%r2
- la %r2,0(%r4,%r1)
-0: srst %r2,%r1
- jo 0b
- brc 13,1f
- slr %r2,%r2
-1: br %r14
-END(memchr)
-libc_hidden_builtin_def (memchr)
diff --git a/sysdeps/s390/s390-32/multiarch/memchr.c b/sysdeps/s390/s390-32/multiarch/memchr.c
deleted file mode 100644
index 5e1610afa4..0000000000
--- a/sysdeps/s390/s390-32/multiarch/memchr.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/* Multiple versions of memchr.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* This wrapper-file is needed, because otherwise file
- sysdeps/s390/s390-[32|64]/memchr.S will be used. */
-#include <sysdeps/s390/multiarch/memchr.c>
diff --git a/sysdeps/s390/s390-64/multiarch/memchr.c b/sysdeps/s390/s390-64/multiarch/memchr.c
deleted file mode 100644
index 5e1610afa4..0000000000
--- a/sysdeps/s390/s390-64/multiarch/memchr.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/* Multiple versions of memchr.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* This wrapper-file is needed, because otherwise file
- sysdeps/s390/s390-[32|64]/memchr.S will be used. */
-#include <sysdeps/s390/multiarch/memchr.c>

[Stefan Liebler]

The ifunc handling for strchr is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strchr variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strchr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strchr.
* sysdeps/s390/multiarch/strchr-c.c: Move to ...
* sysdeps/s390/strchr-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strchr-vx.S: Move to ...
* sysdeps/s390/strchr-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strchr.c: Move to ...
* sysdeps/s390/strchr.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strchr.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strchr.h | 52 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 ++++++-
sysdeps/s390/{multiarch => }/strchr-c.c | 20 +++++----
sysdeps/s390/{multiarch => }/strchr-vx.S | 20 +++++++--
sysdeps/s390/{multiarch => }/strchr.c | 23 ++++++++---
7 files changed, 113 insertions(+), 22 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strchr.h
rename sysdeps/s390/{multiarch => }/strchr-c.c (77%)
rename sysdeps/s390/{multiarch => }/strchr-vx.S (90%)
rename sysdeps/s390/{multiarch => }/strchr.c (71%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 3aaf5a0d40..59d6e845db 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -69,5 +69,6 @@ sysdep_routines += bzero memset memset-z900 \
strcat strcat-vx strcat-c \
strncat strncat-vx strncat-c \
strcmp strcmp-vx strcmp-z900 \
- strncmp strncmp-vx strncmp-c
+ strncmp strncmp-vx strncmp-c \
+ strchr strchr-vx strchr-c
endif
diff --git a/sysdeps/s390/ifunc-strchr.h b/sysdeps/s390/ifunc-strchr.h
new file mode 100644
index 0000000000..cfeb00eeda
--- /dev/null
+++ b/sysdeps/s390/ifunc-strchr.h
@@ -0,0 +1,52 @@
+/* strchr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRCHR_IFUNC 1
+#else
+# define HAVE_STRCHR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRCHR_IFUNC_AND_VX_SUPPORT HAVE_STRCHR_IFUNC
+#else
+# define HAVE_STRCHR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRCHR_DEFAULT STRCHR_Z13
+# define HAVE_STRCHR_C 0
+# define HAVE_STRCHR_Z13 1
+#else
+# define STRCHR_DEFAULT STRCHR_C
+# define HAVE_STRCHR_C 1
+# define HAVE_STRCHR_Z13 HAVE_STRCHR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRCHR_C
+# define STRCHR_C __strchr_c
+#else
+# define STRCHR_C NULL
+#endif
+
+#if HAVE_STRCHR_Z13
+# define STRCHR_Z13 __strchr_vx
+#else
+# define STRCHR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 381376bf9f..a8e9d0acd9 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strchr strchr-vx strchr-c \
- strchrnul strchrnul-vx strchrnul-c \
+sysdep_routines += strchrnul strchrnul-vx strchrnul-c \
strrchr strrchr-vx strrchr-c \
strspn strspn-vx strspn-c \
strpbrk strpbrk-vx strpbrk-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index d982de5788..e809ca3bac 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -36,6 +36,7 @@
#include <ifunc-strncat.h>
#include <ifunc-strcmp.h>
#include <ifunc-strncmp.h>
+#include <ifunc-strchr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -294,6 +295,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRNCMP_IFUNC */

+#if HAVE_STRCHR_IFUNC
+ IFUNC_IMPL (i, name, strchr,
+# if HAVE_STRCHR_Z13
+ IFUNC_IMPL_ADD (array, i, strchr,
+ dl_hwcap & HWCAP_S390_VX, STRCHR_Z13)
+# endif
+# if HAVE_STRCHR_C
+ IFUNC_IMPL_ADD (array, i, strchr, 1, STRCHR_C)
+# endif
+ )
+#endif /* HAVE_STRCHR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -322,7 +335,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcsncmp);

- IFUNC_VX_IMPL (strchr);
IFUNC_VX_IMPL (wcschr);

IFUNC_VX_IMPL (strchrnul);
diff --git a/sysdeps/s390/multiarch/strchr-c.c b/sysdeps/s390/strchr-c.c
similarity index 77%
rename from sysdeps/s390/multiarch/strchr-c.c
rename to sysdeps/s390/strchr-c.c
index 606cb56788..3d3579a1d3 100644
--- a/sysdeps/s390/multiarch/strchr-c.c
+++ b/sysdeps/s390/strchr-c.c
@@ -16,14 +16,18 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRCHR __strchr_c
-# undef weak_alias
-# ifdef SHARED
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
+#include <ifunc-strchr.h>
+
+#if HAVE_STRCHR_C
+# if HAVE_STRCHR_IFUNC
+# define STRCHR STRCHR_C
+# undef weak_alias
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
__hidden_ver1 (__strchr_c, __GI_strchr, __strchr_c);
-# endif /* SHARED */
+# endif
+# endif

# include <string/strchr.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strchr-vx.S b/sysdeps/s390/strchr-vx.S
similarity index 90%
rename from sysdeps/s390/multiarch/strchr-vx.S
rename to sysdeps/s390/strchr-vx.S
index 6e744fb82f..6ffa06f78c 100644
--- a/sysdeps/s390/multiarch/strchr-vx.S
+++ b/sysdeps/s390/strchr-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strchr.h>
+
+#if HAVE_STRCHR_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -36,7 +38,7 @@
-v17=index of unequal
-v18=replicated c
*/
-ENTRY(__strchr_vx)
+ENTRY(STRCHR_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -96,5 +98,15 @@ ENTRY(__strchr_vx)
clije %r3,0,.Lcharacter /* Found zero and c is zero. */
lghi %r2,0 /* Return null if character not found. */
br %r14
-END(__strchr_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRCHR_Z13)
+
+# if ! HAVE_STRCHR_IFUNC
+strong_alias (STRCHR_Z13, strchr)
+weak_alias (strchr, index)
+# endif
+
+# if ! HAVE_STRCHR_C && defined SHARED && IS_IN (libc)
+strong_alias (STRCHR_Z13, __GI_strchr)
+# endif
+
+#endif /* HAVE_STRCHR_Z13 */
diff --git a/sysdeps/s390/multiarch/strchr.c b/sysdeps/s390/strchr.c
similarity index 71%
rename from sysdeps/s390/multiarch/strchr.c
rename to sysdeps/s390/strchr.c
index 8aa33a51cc..a106c6106d 100644
--- a/sysdeps/s390/multiarch/strchr.c
+++ b/sysdeps/s390/strchr.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strchr.h>
+
+#if HAVE_STRCHR_IFUNC
# define strchr __redirect_strchr
/* Omit the strchr inline definitions because it would redefine strchr. */
# define __NO_STRING_INLINES
@@ -24,9 +26,18 @@
# undef strchr
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strchr, __strchr, strchr)
-weak_alias (strchr, index)
+# if HAVE_STRCHR_C
+extern __typeof (__redirect_strchr) STRCHR_C attribute_hidden;
+# endif

-#else
-# include <string/strchr.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_STRCHR_Z13
+extern __typeof (__redirect_strchr) STRCHR_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect_strchr, strchr,
+ (HAVE_STRCHR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRCHR_Z13
+ : STRCHR_DEFAULT
+ )
+weak_alias (strchr, index)
+#endif /* HAVE_STRCHR_IFUNC */

[Stefan Liebler]

The ifunc handling for stpncpy is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove stpncpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add stpncpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for stpncpy.
* sysdeps/s390/multiarch/stpncpy-c.c: Move to ...
* sysdeps/s390/stpncpy-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/stpncpy-vx.S: Move to ...
* sysdeps/s390/stpncpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/stpncpy.c: Move to ...
* sysdeps/s390/stpncpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-stpncpy.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-stpncpy.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 +++++-
sysdeps/s390/{multiarch => }/stpncpy-c.c | 21 +++++----
sysdeps/s390/{multiarch => }/stpncpy-vx.S | 19 +++++++--
sysdeps/s390/{multiarch => }/stpncpy.c | 23 +++++++---
7 files changed, 113 insertions(+), 22 deletions(-)
create mode 100644 sysdeps/s390/ifunc-stpncpy.h
rename sysdeps/s390/{multiarch => }/stpncpy-c.c (74%)
rename sysdeps/s390/{multiarch => }/stpncpy-vx.S (95%)
rename sysdeps/s390/{multiarch => }/stpncpy.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 7cadf7ceba..f02afd4d02 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -64,5 +64,6 @@ sysdep_routines += bzero memset memset-z900 \
strnlen strnlen-vx strnlen-c \
strcpy strcpy-vx strcpy-z900 \
stpcpy stpcpy-vx stpcpy-c \
- strncpy strncpy-vx strncpy-z900
+ strncpy strncpy-vx strncpy-z900 \
+ stpncpy stpncpy-vx stpncpy-c
endif
diff --git a/sysdeps/s390/ifunc-stpncpy.h b/sysdeps/s390/ifunc-stpncpy.h
new file mode 100644
index 0000000000..46e57334e8
--- /dev/null
+++ b/sysdeps/s390/ifunc-stpncpy.h
@@ -0,0 +1,52 @@
+/* stpncpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STPNCPY_IFUNC 1
+#else
+# define HAVE_STPNCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STPNCPY_IFUNC_AND_VX_SUPPORT HAVE_STPNCPY_IFUNC
+#else
+# define HAVE_STPNCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STPNCPY_DEFAULT STPNCPY_Z13
+# define HAVE_STPNCPY_C 0
+# define HAVE_STPNCPY_Z13 1
+#else
+# define STPNCPY_DEFAULT STPNCPY_C
+# define HAVE_STPNCPY_C 1
+# define HAVE_STPNCPY_Z13 HAVE_STPNCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STPNCPY_C
+# define STPNCPY_C __stpncpy_c
+#else
+# define STPNCPY_C NULL
+#endif
+
+#if HAVE_STPNCPY_Z13
+# define STPNCPY_Z13 __stpncpy_vx
+#else
+# define STPNCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index c5189b556c..3d97d21da1 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += stpncpy stpncpy-vx stpncpy-c \
- strcat strcat-vx strcat-c \
+sysdep_routines += strcat strcat-vx strcat-c \
strncat strncat-vx strncat-c \
strcmp strcmp-vx \
strncmp strncmp-vx strncmp-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index d598fc5c22..021e9f247f 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -31,6 +31,7 @@
#include <ifunc-strcpy.h>
#include <ifunc-stpcpy.h>
#include <ifunc-strncpy.h>
+#include <ifunc-stpncpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -229,6 +230,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRNCPY_IFUNC */

+#if HAVE_STPNCPY_IFUNC
+ IFUNC_IMPL (i, name, stpncpy,
+# if HAVE_STPNCPY_Z13
+ IFUNC_IMPL_ADD (array, i, stpncpy,
+ dl_hwcap & HWCAP_S390_VX, STPNCPY_Z13)
+# endif
+# if HAVE_STPNCPY_C
+ IFUNC_IMPL_ADD (array, i, stpncpy, 1, STPNCPY_C)
+# endif
+ )
+#endif /* HAVE_STPNCPY_IFUNC */
+

#ifdef HAVE_S390_VX_ASM_SUPPORT

@@ -248,7 +261,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcsncpy);

- IFUNC_VX_IMPL (stpncpy);
IFUNC_VX_IMPL (wcpncpy);

IFUNC_VX_IMPL (strcat);
diff --git a/sysdeps/s390/multiarch/stpncpy-c.c b/sysdeps/s390/stpncpy-c.c
similarity index 74%
rename from sysdeps/s390/multiarch/stpncpy-c.c
rename to sysdeps/s390/stpncpy-c.c
index 45e50aa9e7..e5d1ae8675 100644
--- a/sysdeps/s390/multiarch/stpncpy-c.c
+++ b/sysdeps/s390/stpncpy-c.c
@@ -16,13 +16,18 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STPNCPY __stpncpy_c
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
- __hidden_ver1 (__stpncpy_c, __GI___stpncpy, __stpncpy_c);
-# endif /* SHARED */
+#include <ifunc-stpncpy.h>
+
+#if HAVE_STPNCPY_C
+# if HAVE_STPNCPY_IFUNC
+# define STPNCPY STPNCPY_C
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__stpncpy_c, __GI___stpncpy, __stpncpy_c);
+# endif
+# endif

# include <string/stpncpy.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/stpncpy-vx.S b/sysdeps/s390/stpncpy-vx.S
similarity index 95%
rename from sysdeps/s390/multiarch/stpncpy-vx.S
rename to sysdeps/s390/stpncpy-vx.S
index 922bd7a355..3dccc10be3 100644
--- a/sysdeps/s390/multiarch/stpncpy-vx.S
+++ b/sysdeps/s390/stpncpy-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-stpncpy.h>
+
+#if HAVE_STPNCPY_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -38,7 +40,7 @@
-%r6 = loaded bytes
-%r7 = border, tmp
*/
-ENTRY(__stpncpy_vx)
+ENTRY(STPNCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -196,5 +198,14 @@ ENTRY(__stpncpy_vx)

vl %v16,0(%r5,%r3) /* Load s. */
j .Llt64
-END(__stpncpy_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STPNCPY_Z13)
+
+# if ! HAVE_STPNCPY_IFUNC
+strong_alias (STPNCPY_Z13, __stpncpy)
+weak_alias (__stpncpy, stpncpy)
+# endif
+
+# if ! HAVE_STPNCPY_C && defined SHARED && IS_IN (libc)
+strong_alias (STPNCPY_Z13, __GI___stpncpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/stpncpy.c b/sysdeps/s390/stpncpy.c
similarity index 70%
rename from sysdeps/s390/multiarch/stpncpy.c
rename to sysdeps/s390/stpncpy.c
index f7f9d51a50..250dc68ed1 100644
--- a/sysdeps/s390/multiarch/stpncpy.c
+++ b/sysdeps/s390/stpncpy.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-stpncpy.h>
+
+#if HAVE_STPNCPY_IFUNC
# define stpncpy __redirect_stpncpy
# define __stpncpy __redirect___stpncpy
# include <string.h>
@@ -24,9 +26,18 @@
# undef __stpncpy
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc_redirected (__redirect___stpncpy, __stpncpy)
-weak_alias (__stpncpy, stpncpy)
+# if HAVE_STPNCPY_C
+extern __typeof (__redirect_stpncpy) STPNCPY_C attribute_hidden;
+# endif

-#else
-# include <string/stpncpy.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_STPNCPY_Z13
+extern __typeof (__redirect_stpncpy) STPNCPY_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect___stpncpy, __stpncpy,
+ (HAVE_STPNCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STPNCPY_Z13
+ : STPNCPY_DEFAULT
+ )
+weak_alias (__stpncpy, stpncpy)
+#endif

[Stefan Liebler]

The ifunc handling for wcsncat is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcsncat variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcsncat variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcsncat.
* sysdeps/s390/multiarch/wcsncat-c.c: Move to ...
* sysdeps/s390/wcsncat-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsncat-vx.S: Move to ...
* sysdeps/s390/wcsncat-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsncat.c: Move to ...
* sysdeps/s390/wcsncat.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcsncat.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcsncat.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcsncat-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcsncat-vx.S | 15 ++++---
sysdeps/s390/{multiarch => }/wcsncat.c | 21 ++++++---
7 files changed, 101 insertions(+), 19 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcsncat.h
rename sysdeps/s390/{multiarch => }/wcsncat-c.c (85%)
rename sysdeps/s390/{multiarch => }/wcsncat-vx.S (97%)
rename sysdeps/s390/{multiarch => }/wcsncat.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index f393a2b1ef..cb8dd2134f 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -89,5 +89,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcpcpy wcpcpy-vx wcpcpy-c \
wcsncpy wcsncpy-vx wcsncpy-c \
wcpncpy wcpncpy-vx wcpncpy-c \
- wcscat wcscat-vx wcscat-c
+ wcscat wcscat-vx wcscat-c \
+ wcsncat wcsncat-vx wcsncat-c
endif
diff --git a/sysdeps/s390/ifunc-wcsncat.h b/sysdeps/s390/ifunc-wcsncat.h
new file mode 100644
index 0000000000..99495e0e64
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcsncat.h
@@ -0,0 +1,53 @@
+/* wcsncat variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSNCAT_IFUNC 1
+#else
+# define HAVE_WCSNCAT_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSNCAT_IFUNC_AND_VX_SUPPORT HAVE_WCSNCAT_IFUNC
+#else
+# define HAVE_WCSNCAT_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSNCAT_DEFAULT WCSNCAT_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSNCAT_C 1
+# define HAVE_WCSNCAT_Z13 1
+#else
+# define WCSNCAT_DEFAULT WCSNCAT_C
+# define HAVE_WCSNCAT_C 1
+# define HAVE_WCSNCAT_Z13 HAVE_WCSNCAT_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSNCAT_C
+# define WCSNCAT_C __wcsncat_c
+#else
+# define WCSNCAT_C NULL
+#endif
+
+#if HAVE_WCSNCAT_Z13
+# define WCSNCAT_Z13 __wcsncat_vx
+#else
+# define WCSNCAT_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 617017496c..6cb75950c2 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcsncat wcsncat-vx wcsncat-c \
- wcscmp wcscmp-vx wcscmp-c \
+sysdep_routines += wcscmp wcscmp-vx wcscmp-c \
wcsncmp wcsncmp-vx wcsncmp-c \
wcschr wcschr-vx wcschr-c \
wcschrnul wcschrnul-vx wcschrnul-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index b05bd35fd8..7b7b1e7497 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -53,6 +53,7 @@
#include <ifunc-wcsncpy.h>
#include <ifunc-wcpncpy.h>
#include <ifunc-wcscat.h>
+#include <ifunc-wcsncat.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -515,6 +516,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSCAT_IFUNC */

+#if HAVE_WCSNCAT_IFUNC
+ IFUNC_IMPL (i, name, wcsncat,
+# if HAVE_WCSNCAT_Z13
+ IFUNC_IMPL_ADD (array, i, wcsncat,
+ dl_hwcap & HWCAP_S390_VX, WCSNCAT_Z13)
+# endif
+# if HAVE_WCSNCAT_C
+ IFUNC_IMPL_ADD (array, i, wcsncat, 1, WCSNCAT_C)
+# endif
+ )
+#endif /* HAVE_WCSNCAT_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -523,8 +536,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcsncat);
-
IFUNC_VX_IMPL (wcscmp);

IFUNC_VX_IMPL (wcsncmp);
diff --git a/sysdeps/s390/multiarch/wcsncat-c.c b/sysdeps/s390/wcsncat-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/wcsncat-c.c
rename to sysdeps/s390/wcsncat-c.c
index 2cf1a76385..5782d5cb28 100644
--- a/sysdeps/s390/multiarch/wcsncat-c.c
+++ b/sysdeps/s390/wcsncat-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSNCAT __wcsncat_c
+#include <ifunc-wcsncat.h>
+
+#if HAVE_WCSNCAT_C
+# if HAVE_WCSNCAT_IFUNC || HAVE_WCSNCAT_Z13
+# define WCSNCAT WCSNCAT_C
+# endif

-# include <wchar.h>
-extern __typeof (wcsncat) __wcsncat_c;
# include <wcsmbs/wcsncat.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcsncat-vx.S b/sysdeps/s390/wcsncat-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/wcsncat-vx.S
rename to sysdeps/s390/wcsncat-vx.S
index 1d3935690d..7c89d3d856 100644
--- a/sysdeps/s390/multiarch/wcsncat-vx.S
+++ b/sysdeps/s390/wcsncat-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsncat.h>
+#if HAVE_WCSNCAT_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -40,7 +41,7 @@
-v18=part of src
-v31=register save area for r6, r7
*/
-ENTRY(__wcsncat_vx)
+ENTRY(WCSNCAT_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -260,6 +261,10 @@ ENTRY(__wcsncat_vx)
j .Lcpy_lt64

.Lfallback:
- jg __wcsncat_c
-END(__wcsncat_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSNCAT_C
+END(WCSNCAT_Z13)
+
+# if ! HAVE_WCSNCAT_IFUNC
+strong_alias (WCSNCAT_Z13, wcsncat)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcsncat.c b/sysdeps/s390/wcsncat.c
similarity index 69%
rename from sysdeps/s390/multiarch/wcsncat.c
rename to sysdeps/s390/wcsncat.c
index c49b8ff786..722429fd5b 100644
--- a/sysdeps/s390/multiarch/wcsncat.c
+++ b/sysdeps/s390/wcsncat.c
@@ -16,12 +16,23 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsncat.h>
+
+#if HAVE_WCSNCAT_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2 (__wcsncat, wcsncat)
+# if HAVE_WCSNCAT_C
+extern __typeof (wcsncat) WCSNCAT_C attribute_hidden;
+# endif
+
+# if HAVE_WCSNCAT_Z13
+extern __typeof (wcsncat) WCSNCAT_Z13 attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcsncat.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (wcsncat, wcsncat,
+ (HAVE_WCSNCAT_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSNCAT_Z13
+ : WCSNCAT_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for strcmp is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

Note: The fallback s390-32/s390-64 ifunc variants with clst instruction
are now moved to the unified strcmp-z900.S file which can be used for
31/64bit. The s390-32/s390-64 files multiarch/strcmp.c and strcmp.S
are deleted.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strcmp variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strcmp variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strcmp.
* sysdeps/s390/multiarch/strcmp-vx.S: Move to ...
* sysdeps/s390/strcmp-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strcmp.c: Move to ...
* sysdeps/s390/strcmp.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strcmp.h: New file.
* sysdeps/s390/s390-64/strcmp.S: Move to ...
* sysdeps/s390/strcmp-z900.S: ... here and adjust to be usable
for 31/64bit and ifunc handling.
* sysdeps/s390/s390-32/multiarch/strcmp.c: Delete file.
* sysdeps/s390/s390-64/multiarch/strcmp.c: Likewise.
* sysdeps/s390/s390-32/strcmp.S: Likewise.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strcmp.h | 52 +++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 ++++-
sysdeps/s390/s390-32/multiarch/strcmp.c | 21 --------
sysdeps/s390/s390-32/strcmp.S | 41 ---------------
sysdeps/s390/s390-64/multiarch/strcmp.c | 21 --------
sysdeps/s390/{multiarch => }/strcmp-vx.S | 19 ++++---
.../s390/{s390-64/strcmp.S => strcmp-z900.S} | 32 +++++++++---
sysdeps/s390/{multiarch => }/strcmp.c | 17 +++++-
10 files changed, 119 insertions(+), 104 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strcmp.h
delete mode 100644 sysdeps/s390/s390-32/multiarch/strcmp.c
delete mode 100644 sysdeps/s390/s390-32/strcmp.S
delete mode 100644 sysdeps/s390/s390-64/multiarch/strcmp.c
rename sysdeps/s390/{multiarch => }/strcmp-vx.S (90%)
rename sysdeps/s390/{s390-64/strcmp.S => strcmp-z900.S} (70%)
rename sysdeps/s390/{multiarch => }/strcmp.c (71%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index ec0fb8c2ad..9e7d5625c9 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -67,5 +67,6 @@ sysdep_routines += bzero memset memset-z900 \
strncpy strncpy-vx strncpy-z900 \
stpncpy stpncpy-vx stpncpy-c \
strcat strcat-vx strcat-c \
- strncat strncat-vx strncat-c
+ strncat strncat-vx strncat-c \
+ strcmp strcmp-vx strcmp-z900
endif
diff --git a/sysdeps/s390/ifunc-strcmp.h b/sysdeps/s390/ifunc-strcmp.h
new file mode 100644
index 0000000000..86ffe686ad
--- /dev/null
+++ b/sysdeps/s390/ifunc-strcmp.h
@@ -0,0 +1,52 @@
+/* strcmp variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRCMP_IFUNC 1
+#else
+# define HAVE_STRCMP_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRCMP_IFUNC_AND_VX_SUPPORT HAVE_STRCMP_IFUNC
+#else
+# define HAVE_STRCMP_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRCMP_DEFAULT STRCMP_Z13
+# define HAVE_STRCMP_Z900_G5 0
+# define HAVE_STRCMP_Z13 1
+#else
+# define STRCMP_DEFAULT STRCMP_Z900_G5
+# define HAVE_STRCMP_Z900_G5 1
+# define HAVE_STRCMP_Z13 HAVE_STRCMP_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRCMP_Z900_G5
+# define STRCMP_Z900_G5 __strcmp_default
+#else
+# define STRCMP_Z900_G5 NULL
+#endif
+
+#if HAVE_STRCMP_Z13
+# define STRCMP_Z13 __strcmp_vx
+#else
+# define STRCMP_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 24be3eac51..97421a4996 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strcmp strcmp-vx \
- strncmp strncmp-vx strncmp-c \
+sysdep_routines += strncmp strncmp-vx strncmp-c \
strchr strchr-vx strchr-c \
strchrnul strchrnul-vx strchrnul-c \
strrchr strrchr-vx strrchr-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 3abcaf08e0..44637c431b 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -34,6 +34,7 @@
#include <ifunc-stpncpy.h>
#include <ifunc-strcat.h>
#include <ifunc-strncat.h>
+#include <ifunc-strcmp.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -268,6 +269,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRNCAT_IFUNC */

+#if HAVE_STRCMP_IFUNC
+ IFUNC_IMPL (i, name, strcmp,
+# if HAVE_STRCMP_Z13
+ IFUNC_IMPL_ADD (array, i, strcmp,
+ dl_hwcap & HWCAP_S390_VX, STRCMP_Z13)
+# endif
+# if HAVE_STRCMP_Z900_G5
+ IFUNC_IMPL_ADD (array, i, strcmp, 1, STRCMP_Z900_G5)
+# endif
+ )
+#endif /* HAVE_STRCMP_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -292,7 +305,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcsncat);

- IFUNC_VX_IMPL (strcmp);
IFUNC_VX_IMPL (wcscmp);

IFUNC_VX_IMPL (strncmp);
diff --git a/sysdeps/s390/s390-32/multiarch/strcmp.c b/sysdeps/s390/s390-32/multiarch/strcmp.c
deleted file mode 100644
index d06b0f3436..0000000000
--- a/sysdeps/s390/s390-32/multiarch/strcmp.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/* Multiple versions of strcmp.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* This wrapper-file is needed, because otherwise file
- sysdeps/s390/s390-[32|64]/strcmp.S will be used. */
-#include <sysdeps/s390/multiarch/strcmp.c>
diff --git a/sysdeps/s390/s390-32/strcmp.S b/sysdeps/s390/s390-32/strcmp.S
deleted file mode 100644
index 3cf3f239fd..0000000000
--- a/sysdeps/s390/s390-32/strcmp.S
+++ /dev/null
@@ -1,41 +0,0 @@
-/* strcmp - compare two string. S/390 version.
- This file is part of the GNU C Library.
- Copyright (C) 2001-2018 Free Software Foundation, Inc.
- Contributed by Martin Schwidefsky (***@de.ibm.com).
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* INPUT PARAMETERS
- %r2 = address of string 1
- %r3 = address of string 2. */
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
- .text
-ENTRY(strcmp)
- slr %r0,%r0
-0: clst %r2,%r3
- jo 0b
- jp 1f
- jm 2f
- slr %r2,%r2
- br %r14
-1: lhi %r2,1
- br %r14
-2: lhi %r2,-1
- br %r14
-END(strcmp)
-libc_hidden_builtin_def (strcmp)
diff --git a/sysdeps/s390/s390-64/multiarch/strcmp.c b/sysdeps/s390/s390-64/multiarch/strcmp.c
deleted file mode 100644
index d06b0f3436..0000000000
--- a/sysdeps/s390/s390-64/multiarch/strcmp.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/* Multiple versions of strcmp.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* This wrapper-file is needed, because otherwise file
- sysdeps/s390/s390-[32|64]/strcmp.S will be used. */
-#include <sysdeps/s390/multiarch/strcmp.c>
diff --git a/sysdeps/s390/multiarch/strcmp-vx.S b/sysdeps/s390/strcmp-vx.S
similarity index 90%
rename from sysdeps/s390/multiarch/strcmp-vx.S
rename to sysdeps/s390/strcmp-vx.S
index bcaeb564d4..801ad9d32b 100644
--- a/sysdeps/s390/multiarch/strcmp-vx.S
+++ b/sysdeps/s390/strcmp-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strcmp.h>
+#if HAVE_STRCMP_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -36,7 +37,7 @@
-v17=part of s2
-v18=index of unequal
*/
-ENTRY(__strcmp_vx)
+ENTRY(STRCMP_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -106,11 +107,13 @@ ENTRY(__strcmp_vx)
.Lend_equal:
lghi %r2,0
br %r14
-END(__strcmp_vx)
+END(STRCMP_Z13)

-# define strcmp __strcmp_c
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) strong_alias(__strcmp_c, __GI_strcmp)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+# if ! HAVE_STRCMP_IFUNC
+strong_alias (STRCMP_Z13, strcmp)
+# endif

-#include <strcmp.S>
+# if ! HAVE_STRCMP_Z900_G5 && defined SHARED && IS_IN (libc)
+strong_alias (STRCMP_Z13, __GI_strcmp)
+# endif
+#endif
diff --git a/sysdeps/s390/s390-64/strcmp.S b/sysdeps/s390/strcmp-z900.S
similarity index 70%
rename from sysdeps/s390/s390-64/strcmp.S
rename to sysdeps/s390/strcmp-z900.S
index 6cf1addd8b..67b3c8b2e5 100644
--- a/sysdeps/s390/s390-64/strcmp.S
+++ b/sysdeps/s390/strcmp-z900.S
@@ -21,21 +21,39 @@
%r2 = address of string 1
%r3 = address of string 2. */

+#include <ifunc-strcmp.h>
#include "sysdep.h"
#include "asm-syntax.h"

+#if HAVE_STRCMP_Z900_G5
+# if defined __s390x__
+# define SLGR slgr
+# define LGHI lghi
+# else
+# define SLGR slr
+# define LGHI lhi
+# endif /* ! defined __s390x__ */
+
.text
-ENTRY(strcmp)
- slr %r0,%r0
+ENTRY(STRCMP_Z900_G5)
+ SLGR %r0,%r0
0: clst %r2,%r3
jo 0b
jp 1f
jm 2f
- slgr %r2,%r2
+ SLGR %r2,%r2
br %r14
-1: lghi %r2,1
+1: LGHI %r2,1
br %r14
-2: lghi %r2,-1
+2: LGHI %r2,-1
br %r14
-END(strcmp)
-libc_hidden_builtin_def (strcmp)
+END(STRCMP_Z900_G5)
+
+# if ! HAVE_STRCMP_IFUNC
+strong_alias (STRCMP_Z900_G5, strcmp)
+# endif
+
+# if defined SHARED && IS_IN (libc)
+strong_alias (STRCMP_Z900_G5, __GI_strcmp)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/strcmp.c b/sysdeps/s390/strcmp.c
similarity index 71%
rename from sysdeps/s390/multiarch/strcmp.c
rename to sysdeps/s390/strcmp.c
index 7c8b17b304..9efa30acaf 100644
--- a/sysdeps/s390/multiarch/strcmp.c
+++ b/sysdeps/s390/strcmp.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strcmp.h>
+
+#if HAVE_STRCMP_IFUNC
# define strcmp __redirect_strcmp
/* Omit the strcmp inline definitions because it would redefine strcmp. */
# define __NO_STRING_INLINES
@@ -24,6 +26,17 @@
# include <ifunc-resolve.h>
# undef strcmp

-s390_vx_libc_ifunc2_redirected (__redirect_strcmp, __strcmp, strcmp)
+# if HAVE_STRCMP_Z900_G5
+extern __typeof (__redirect_strcmp) STRCMP_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_STRCMP_Z13
+extern __typeof (__redirect_strcmp) STRCMP_Z13 attribute_hidden;
+# endif

+s390_libc_ifunc_expr (__redirect_strcmp, strcmp,
+ (HAVE_STRCMP_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRCMP_Z13
+ : STRCMP_DEFAULT
+ )
#endif

[Stefan Liebler]

The ifunc handling for strpbrk is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strpbrk variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strpbrk variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strpbrk.
* sysdeps/s390/multiarch/strpbrk-c.c: Move to ...
* sysdeps/s390/strpbrk-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strpbrk-vx.S: Move to ...
* sysdeps/s390/strpbrk-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strpbrk.c: Move to ...
* sysdeps/s390/strpbrk.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strpbrk.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strpbrk.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 +++++-
sysdeps/s390/{multiarch => }/strpbrk-c.c | 20 +++++----
sysdeps/s390/{multiarch => }/strpbrk-vx.S | 19 +++++++--
sysdeps/s390/{multiarch => }/strpbrk.c | 21 ++++++---
7 files changed, 111 insertions(+), 21 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strpbrk.h
rename sysdeps/s390/{multiarch => }/strpbrk-c.c (73%)
rename sysdeps/s390/{multiarch => }/strpbrk-vx.S (97%)
rename sysdeps/s390/{multiarch => }/strpbrk.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 604ca68ef1..df1f858e28 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -73,5 +73,6 @@ sysdep_routines += bzero memset memset-z900 \
strchr strchr-vx strchr-c \
strchrnul strchrnul-vx strchrnul-c \
strrchr strrchr-vx strrchr-c \
- strspn strspn-vx strspn-c
+ strspn strspn-vx strspn-c \
+ strpbrk strpbrk-vx strpbrk-c
endif
diff --git a/sysdeps/s390/ifunc-strpbrk.h b/sysdeps/s390/ifunc-strpbrk.h
new file mode 100644
index 0000000000..4a3138c6bf
--- /dev/null
+++ b/sysdeps/s390/ifunc-strpbrk.h
@@ -0,0 +1,52 @@
+/* strpbrk variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRPBRK_IFUNC 1
+#else
+# define HAVE_STRPBRK_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRPBRK_IFUNC_AND_VX_SUPPORT HAVE_STRPBRK_IFUNC
+#else
+# define HAVE_STRPBRK_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRPBRK_DEFAULT STRPBRK_Z13
+# define HAVE_STRPBRK_C 0
+# define HAVE_STRPBRK_Z13 1
+#else
+# define STRPBRK_DEFAULT STRPBRK_C
+# define HAVE_STRPBRK_C 1
+# define HAVE_STRPBRK_Z13 HAVE_STRPBRK_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRPBRK_C
+# define STRPBRK_C __strpbrk_c
+#else
+# define STRPBRK_C NULL
+#endif
+
+#if HAVE_STRPBRK_Z13
+# define STRPBRK_Z13 __strpbrk_vx
+#else
+# define STRPBRK_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 9b141e338c..1a3fed9fc8 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strpbrk strpbrk-vx strpbrk-c \
- strcspn strcspn-vx strcspn-c \
+sysdep_routines += strcspn strcspn-vx strcspn-c \
memchr memchr-vx \
rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index c39e1f793a..8e23416730 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -40,6 +40,7 @@
#include <ifunc-strchrnul.h>
#include <ifunc-strrchr.h>
#include <ifunc-strspn.h>
+#include <ifunc-strpbrk.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -346,6 +347,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRSPN_IFUNC */

+#if HAVE_STRPBRK_IFUNC
+ IFUNC_IMPL (i, name, strpbrk,
+# if HAVE_STRPBRK_Z13
+ IFUNC_IMPL_ADD (array, i, strpbrk,
+ dl_hwcap & HWCAP_S390_VX, STRPBRK_Z13)
+# endif
+# if HAVE_STRPBRK_C
+ IFUNC_IMPL_ADD (array, i, strpbrk, 1, STRPBRK_C)
+# endif
+ )
+#endif /* HAVE_STRPBRK_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -382,7 +395,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcsspn);

- IFUNC_VX_IMPL (strpbrk);
IFUNC_VX_IMPL (wcspbrk);

IFUNC_VX_IMPL (strcspn);
diff --git a/sysdeps/s390/multiarch/strpbrk-c.c b/sysdeps/s390/strpbrk-c.c
similarity index 73%
rename from sysdeps/s390/multiarch/strpbrk-c.c
rename to sysdeps/s390/strpbrk-c.c
index 2c0517aeb5..70cc5db672 100644
--- a/sysdeps/s390/multiarch/strpbrk-c.c
+++ b/sysdeps/s390/strpbrk-c.c
@@ -16,13 +16,17 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRPBRK __strpbrk_c
-# ifdef SHARED
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
- __hidden_ver1 (__strpbrk_c, __GI_strpbrk, __strpbrk_c);
-# endif /* SHARED */
+#include <ifunc-strpbrk.h>
+
+#if HAVE_STRPBRK_C
+# if HAVE_STRPBRK_IFUNC
+# define STRPBRK STRPBRK_C
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strpbrk_c, __GI_strpbrk, __strpbrk_c);
+# endif
+# endif

# include <string/strpbrk.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strpbrk-vx.S b/sysdeps/s390/strpbrk-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/strpbrk-vx.S
rename to sysdeps/s390/strpbrk-vx.S
index e19c550ed4..0fc7dc1433 100644
--- a/sysdeps/s390/multiarch/strpbrk-vx.S
+++ b/sysdeps/s390/strpbrk-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strpbrk.h>
+
+#if HAVE_STRPBRK_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -59,7 +61,7 @@
otherwise =0;
r9: loaded byte count of vlbb accept-string
*/
-ENTRY(__strpbrk_vx)
+ENTRY(STRPBRK_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -298,5 +300,14 @@ ENTRY(__strpbrk_vx)
vlgvg %r9,%v31,1
lgr %r2,%r1
br %r14
-END(__strpbrk_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRPBRK_Z13)
+
+# if ! HAVE_STRPBRK_IFUNC
+strong_alias (STRPBRK_Z13, strpbrk)
+# endif
+
+# if ! HAVE_STRPBRK_C && defined SHARED && IS_IN (libc)
+strong_alias (STRPBRK_Z13, __GI_strpbrk)
+# endif
+
+#endif /* HAVE_STRPBRK_Z13 */
diff --git a/sysdeps/s390/multiarch/strpbrk.c b/sysdeps/s390/strpbrk.c
similarity index 70%
rename from sysdeps/s390/multiarch/strpbrk.c
rename to sysdeps/s390/strpbrk.c
index 11afc268f7..41ce00a1ae 100644
--- a/sysdeps/s390/multiarch/strpbrk.c
+++ b/sysdeps/s390/strpbrk.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strpbrk.h>
+
+#if HAVE_STRPBRK_IFUNC
# define strpbrk __redirect_strpbrk
/* Omit the strpbrk inline definitions because it would redefine strpbrk. */
# define __NO_STRING_INLINES
@@ -24,8 +26,17 @@
# undef strpbrk
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strpbrk, __strpbrk, strpbrk)
+# if HAVE_STRPBRK_C
+extern __typeof (__redirect_strpbrk) STRPBRK_C attribute_hidden;
+# endif
+
+# if HAVE_STRPBRK_Z13
+extern __typeof (__redirect_strpbrk) STRPBRK_Z13 attribute_hidden;
+# endif

-#else
-# include <string/strpbrk.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_strpbrk, strpbrk,
+ (HAVE_STRPBRK_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRPBRK_Z13
+ : STRPBRK_DEFAULT
+ )
+#endif /* HAVE_STRPBRK_IFUNC */

[Stefan Liebler]

The ifunc handling for wcsnlen is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.
Glibc internal calls will use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcsnlen variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcsnlen variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcsnlen.
* sysdeps/s390/multiarch/wcsnlen-c.c: Move to ...
* sysdeps/s390/wcsnlen-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsnlen-vx.S: Move to ...
* sysdeps/s390/wcsnlen-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsnlen.c: Move to ...
* sysdeps/s390/wcsnlen.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcsnlen.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcsnlen.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcsnlen-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcsnlen-vx.S | 16 ++++---
sysdeps/s390/{multiarch => }/wcsnlen.c | 23 +++++++---
7 files changed, 103 insertions(+), 20 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcsnlen.h
rename sysdeps/s390/{multiarch => }/wcsnlen-c.c (85%)
rename sysdeps/s390/{multiarch => }/wcsnlen-vx.S (95%)
rename sysdeps/s390/{multiarch => }/wcsnlen.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 10c0e1bea0..590d2207b7 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -83,5 +83,6 @@ sysdep_routines += bzero memset memset-z900 \
endif

ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcslen wcslen-vx wcslen-c
+sysdep_routines += wcslen wcslen-vx wcslen-c \
+ wcsnlen wcsnlen-vx wcsnlen-c
endif
diff --git a/sysdeps/s390/ifunc-wcsnlen.h b/sysdeps/s390/ifunc-wcsnlen.h
new file mode 100644
index 0000000000..b5b21da2f1
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcsnlen.h
@@ -0,0 +1,53 @@
+/* wcsnlen variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSNLEN_IFUNC 1
+#else
+# define HAVE_WCSNLEN_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSNLEN_IFUNC_AND_VX_SUPPORT HAVE_WCSNLEN_IFUNC
+#else
+# define HAVE_WCSNLEN_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSNLEN_DEFAULT WCSNLEN_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSNLEN_C 1
+# define HAVE_WCSNLEN_Z13 1
+#else
+# define WCSNLEN_DEFAULT WCSNLEN_C
+# define HAVE_WCSNLEN_C 1
+# define HAVE_WCSNLEN_Z13 HAVE_WCSNLEN_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSNLEN_C
+# define WCSNLEN_C __wcsnlen_c
+#else
+# define WCSNLEN_C NULL
+#endif
+
+#if HAVE_WCSNLEN_Z13
+# define WCSNLEN_Z13 __wcsnlen_vx
+#else
+# define WCSNLEN_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 421d40d020..ce2e7ce5f4 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcsnlen wcsnlen-vx wcsnlen-c \
- wcscpy wcscpy-vx wcscpy-c \
+sysdep_routines += wcscpy wcscpy-vx wcscpy-c \
wcpcpy wcpcpy-vx wcpcpy-c \
wcsncpy wcsncpy-vx wcsncpy-c \
wcpncpy wcpncpy-vx wcpncpy-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 7bf5f14c01..c199fd0e0b 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -47,6 +47,7 @@
#include <ifunc-memccpy.h>
#include <ifunc-memrchr.h>
#include <ifunc-wcslen.h>
+#include <ifunc-wcsnlen.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -437,6 +438,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSLEN_IFUNC */

+#if HAVE_WCSNLEN_IFUNC
+ IFUNC_IMPL (i, name, wcsnlen,
+# if HAVE_WCSNLEN_Z13
+ IFUNC_IMPL_ADD (array, i, wcsnlen,
+ dl_hwcap & HWCAP_S390_VX, WCSNLEN_Z13)
+# endif
+# if HAVE_WCSNLEN_C
+ IFUNC_IMPL_ADD (array, i, wcsnlen, 1, WCSNLEN_C)
+# endif
+ )
+#endif /* HAVE_WCSNLEN_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -445,8 +458,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcsnlen);
-
IFUNC_VX_IMPL (wcscpy);

IFUNC_VX_IMPL (wcpcpy);
diff --git a/sysdeps/s390/multiarch/wcsnlen-c.c b/sysdeps/s390/wcsnlen-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/wcsnlen-c.c
rename to sysdeps/s390/wcsnlen-c.c
index 8f43f5104f..7495a6f97c 100644
--- a/sysdeps/s390/multiarch/wcsnlen-c.c
+++ b/sysdeps/s390/wcsnlen-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSNLEN __wcsnlen_c
+#include <ifunc-wcsnlen.h>
+
+#if HAVE_WCSNLEN_C
+# if HAVE_WCSNLEN_IFUNC || HAVE_WCSNLEN_Z13
+# define WCSNLEN WCSNLEN_C
+# endif

-# include <wchar.h>
-extern __typeof (__wcsnlen) __wcsnlen_c;
# include <wcsmbs/wcsnlen.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcsnlen-vx.S b/sysdeps/s390/wcsnlen-vx.S
similarity index 95%
rename from sysdeps/s390/multiarch/wcsnlen-vx.S
rename to sysdeps/s390/wcsnlen-vx.S
index 420f29fbc0..47f4ca8284 100644
--- a/sysdeps/s390/multiarch/wcsnlen-vx.S
+++ b/sysdeps/s390/wcsnlen-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsnlen.h>
+#if HAVE_WCSNLEN_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -34,7 +35,7 @@
-r5=current_len and return_value
-v16=part of s
*/
-ENTRY(__wcsnlen_vx)
+ENTRY(WCSNLEN_Z13)

.machine "z13"
.machinemode "zarch_nohighgprs"
@@ -146,6 +147,11 @@ ENTRY(__wcsnlen_vx)
j .Llt64

.Lfallback:
- jg __wcsnlen_c
-END(__wcsnlen_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSNLEN_C
+END(WCSNLEN_Z13)
+
+# if ! HAVE_WCSNLEN_IFUNC
+strong_alias (WCSNLEN_Z13, __wcsnlen)
+weak_alias (__wcsnlen, wcsnlen)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcsnlen.c b/sysdeps/s390/wcsnlen.c
similarity index 70%
rename from sysdeps/s390/multiarch/wcsnlen.c
rename to sysdeps/s390/wcsnlen.c
index 5234074b1f..b5c8ad9fde 100644
--- a/sysdeps/s390/multiarch/wcsnlen.c
+++ b/sysdeps/s390/wcsnlen.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsnlen.h>
+
+#if HAVE_WCSNLEN_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__wcsnlen)
-weak_alias (__wcsnlen, wcsnlen)
+# if HAVE_WCSNLEN_C
+extern __typeof (__wcsnlen) WCSNLEN_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcsnlen.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCSNLEN_Z13
+extern __typeof (__wcsnlen) WCSNLEN_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__wcsnlen, __wcsnlen,
+ (HAVE_WCSNLEN_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSNLEN_Z13
+ : WCSNLEN_DEFAULT
+ )
+weak_alias (__wcsnlen, wcsnlen)
+#endif

[Stefan Liebler]

The ifunc handling for wcpncpy is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcpncpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcpncpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcpncpy.
* sysdeps/s390/multiarch/wcpncpy-c.c: Move to ...
* sysdeps/s390/wcpncpy-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcpncpy-vx.S: Move to ...
* sysdeps/s390/wcpncpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcpncpy.c: Move to ...
* sysdeps/s390/wcpncpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcpncpy.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcpncpy.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcpncpy-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcpncpy-vx.S | 16 ++++---
sysdeps/s390/{multiarch => }/wcpncpy.c | 23 +++++++---
7 files changed, 103 insertions(+), 20 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcpncpy.h
rename sysdeps/s390/{multiarch => }/wcpncpy-c.c (85%)
rename sysdeps/s390/{multiarch => }/wcpncpy-vx.S (97%)
rename sysdeps/s390/{multiarch => }/wcpncpy.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index e10ad5b470..e49b1bd26b 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -87,5 +87,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsnlen wcsnlen-vx wcsnlen-c \
wcscpy wcscpy-vx wcscpy-c \
wcpcpy wcpcpy-vx wcpcpy-c \
- wcsncpy wcsncpy-vx wcsncpy-c
+ wcsncpy wcsncpy-vx wcsncpy-c \
+ wcpncpy wcpncpy-vx wcpncpy-c
endif
diff --git a/sysdeps/s390/ifunc-wcpncpy.h b/sysdeps/s390/ifunc-wcpncpy.h
new file mode 100644
index 0000000000..0dd5633aa9
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcpncpy.h
@@ -0,0 +1,53 @@
+/* wcpncpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCPNCPY_IFUNC 1
+#else
+# define HAVE_WCPNCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCPNCPY_IFUNC_AND_VX_SUPPORT HAVE_WCPNCPY_IFUNC
+#else
+# define HAVE_WCPNCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCPNCPY_DEFAULT WCPNCPY_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCPNCPY_C 1
+# define HAVE_WCPNCPY_Z13 1
+#else
+# define WCPNCPY_DEFAULT WCPNCPY_C
+# define HAVE_WCPNCPY_C 1
+# define HAVE_WCPNCPY_Z13 HAVE_WCPNCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCPNCPY_C
+# define WCPNCPY_C __wcpncpy_c
+#else
+# define WCPNCPY_C NULL
+#endif
+
+#if HAVE_WCPNCPY_Z13
+# define WCPNCPY_Z13 __wcpncpy_vx
+#else
+# define WCPNCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 6631fd14d3..158fb49552 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcpncpy wcpncpy-vx wcpncpy-c \
- wcscat wcscat-vx wcscat-c \
+sysdep_routines += wcscat wcscat-vx wcscat-c \
wcsncat wcsncat-vx wcsncat-c \
wcscmp wcscmp-vx wcscmp-c \
wcsncmp wcsncmp-vx wcsncmp-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 9ebaf4de6f..e60238fcde 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -51,6 +51,7 @@
#include <ifunc-wcscpy.h>
#include <ifunc-wcpcpy.h>
#include <ifunc-wcsncpy.h>
+#include <ifunc-wcpncpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -489,6 +490,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSNCPY_IFUNC */

+#if HAVE_WCPNCPY_IFUNC
+ IFUNC_IMPL (i, name, wcpncpy,
+# if HAVE_WCPNCPY_Z13
+ IFUNC_IMPL_ADD (array, i, wcpncpy,
+ dl_hwcap & HWCAP_S390_VX, WCPNCPY_Z13)
+# endif
+# if HAVE_WCPNCPY_C
+ IFUNC_IMPL_ADD (array, i, wcpncpy, 1, WCPNCPY_C)
+# endif
+ )
+#endif /* HAVE_WCPNCPY_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -497,8 +510,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcpncpy);
-
IFUNC_VX_IMPL (wcscat);

IFUNC_VX_IMPL (wcsncat);
diff --git a/sysdeps/s390/multiarch/wcpncpy-c.c b/sysdeps/s390/wcpncpy-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/wcpncpy-c.c
rename to sysdeps/s390/wcpncpy-c.c
index 1f44bacea9..d03359217e 100644
--- a/sysdeps/s390/multiarch/wcpncpy-c.c
+++ b/sysdeps/s390/wcpncpy-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCPNCPY __wcpncpy_c
+#include <ifunc-wcpncpy.h>
+
+#if HAVE_WCPNCPY_C
+# if HAVE_WCPNCPY_IFUNC || HAVE_WCPNCPY_Z13
+# define WCPNCPY WCPNCPY_C
+# endif

-# include <wchar.h>
-extern __typeof (__wcpncpy) __wcpncpy_c;
# include <wcsmbs/wcpncpy.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcpncpy-vx.S b/sysdeps/s390/wcpncpy-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/wcpncpy-vx.S
rename to sysdeps/s390/wcpncpy-vx.S
index 004f512e1f..7b5e4ad32d 100644
--- a/sysdeps/s390/multiarch/wcpncpy-vx.S
+++ b/sysdeps/s390/wcpncpy-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcpncpy.h>
+#if HAVE_WCPNCPY_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -38,7 +39,7 @@
-%r6 = loaded bytes
-%r7 = border, tmp
*/
-ENTRY(__wcpncpy_vx)
+ENTRY(WCPNCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -217,6 +218,11 @@ ENTRY(__wcpncpy_vx)
j .Llt64

.Lfallback:
- jg __wcpncpy_c
-END(__wcpncpy_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCPNCPY_C
+END(WCPNCPY_Z13)
+
+# if ! HAVE_WCPNCPY_IFUNC
+strong_alias (WCPNCPY_Z13, __wcpncpy)
+weak_alias (__wcpncpy, wcpncpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcpncpy.c b/sysdeps/s390/wcpncpy.c
similarity index 70%
rename from sysdeps/s390/multiarch/wcpncpy.c
rename to sysdeps/s390/wcpncpy.c
index b72265fbe9..08d097dd2c 100644
--- a/sysdeps/s390/multiarch/wcpncpy.c
+++ b/sysdeps/s390/wcpncpy.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcpncpy.h>
+
+#if HAVE_WCPNCPY_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__wcpncpy)
-weak_alias (__wcpncpy, wcpncpy)
+# if HAVE_WCPNCPY_C
+extern __typeof (__wcpncpy) WCPNCPY_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcpncpy.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCPNCPY_Z13
+extern __typeof (__wcpncpy) WCPNCPY_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__wcpncpy, __wcpncpy,
+ (HAVE_WCPNCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCPNCPY_Z13
+ : WCPNCPY_DEFAULT
+ )
+weak_alias (__wcpncpy, wcpncpy)
+#endif

[Stefan Liebler]

The ifunc handling for strrchr is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strrchr variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strrchr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strrchr.
* sysdeps/s390/multiarch/strrchr-c.c: Move to ...
* sysdeps/s390/strrchr-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strrchr-vx.S: Move to ...
* sysdeps/s390/strrchr-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strrchr.c: Move to ...
* sysdeps/s390/strrchr.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strrchr.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strrchr.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 +++++-
sysdeps/s390/{multiarch => }/strrchr-c.c | 20 +++++----
sysdeps/s390/{multiarch => }/strrchr-vx.S | 20 +++++++--
sysdeps/s390/{multiarch => }/strrchr.c | 23 +++++++---
7 files changed, 113 insertions(+), 22 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strrchr.h
rename sysdeps/s390/{multiarch => }/strrchr-c.c (77%)
rename sysdeps/s390/{multiarch => }/strrchr-vx.S (94%)
rename sysdeps/s390/{multiarch => }/strrchr.c (66%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 07ec5e969f..c423c80a43 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -71,5 +71,6 @@ sysdep_routines += bzero memset memset-z900 \
strcmp strcmp-vx strcmp-z900 \
strncmp strncmp-vx strncmp-c \
strchr strchr-vx strchr-c \
- strchrnul strchrnul-vx strchrnul-c
+ strchrnul strchrnul-vx strchrnul-c \
+ strrchr strrchr-vx strrchr-c
endif
diff --git a/sysdeps/s390/ifunc-strrchr.h b/sysdeps/s390/ifunc-strrchr.h
new file mode 100644
index 0000000000..7185fc3260
--- /dev/null
+++ b/sysdeps/s390/ifunc-strrchr.h
@@ -0,0 +1,52 @@
+/* strrchr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRRCHR_IFUNC 1
+#else
+# define HAVE_STRRCHR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRRCHR_IFUNC_AND_VX_SUPPORT HAVE_STRRCHR_IFUNC
+#else
+# define HAVE_STRRCHR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRRCHR_DEFAULT STRRCHR_Z13
+# define HAVE_STRRCHR_C 0
+# define HAVE_STRRCHR_Z13 1
+#else
+# define STRRCHR_DEFAULT STRRCHR_C
+# define HAVE_STRRCHR_C 1
+# define HAVE_STRRCHR_Z13 HAVE_STRRCHR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRRCHR_C
+# define STRRCHR_C __strrchr_c
+#else
+# define STRRCHR_C NULL
+#endif
+
+#if HAVE_STRRCHR_Z13
+# define STRRCHR_Z13 __strrchr_vx
+#else
+# define STRRCHR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 999a979fee..c826755558 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strrchr strrchr-vx strrchr-c \
- strspn strspn-vx strspn-c \
+sysdep_routines += strspn strspn-vx strspn-c \
strpbrk strpbrk-vx strpbrk-c \
strcspn strcspn-vx strcspn-c \
memchr memchr-vx \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 0a47ffeac3..60cd705ffa 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -38,6 +38,7 @@
#include <ifunc-strncmp.h>
#include <ifunc-strchr.h>
#include <ifunc-strchrnul.h>
+#include <ifunc-strrchr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -320,6 +321,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRCHRNUL_IFUNC */

+#if HAVE_STRRCHR_IFUNC
+ IFUNC_IMPL (i, name, strrchr,
+# if HAVE_STRRCHR_Z13
+ IFUNC_IMPL_ADD (array, i, strrchr,
+ dl_hwcap & HWCAP_S390_VX, STRRCHR_Z13)
+# endif
+# if HAVE_STRRCHR_C
+ IFUNC_IMPL_ADD (array, i, strrchr, 1, STRRCHR_C)
+# endif
+ )
+#endif /* HAVE_STRRCHR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -352,7 +365,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcschrnul);

- IFUNC_VX_IMPL (strrchr);
IFUNC_VX_IMPL (wcsrchr);

IFUNC_VX_IMPL (strspn);
diff --git a/sysdeps/s390/multiarch/strrchr-c.c b/sysdeps/s390/strrchr-c.c
similarity index 77%
rename from sysdeps/s390/multiarch/strrchr-c.c
rename to sysdeps/s390/strrchr-c.c
index 53ceb8086f..615f16da7d 100644
--- a/sysdeps/s390/multiarch/strrchr-c.c
+++ b/sysdeps/s390/strrchr-c.c
@@ -16,14 +16,18 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRRCHR __strrchr_c
-# undef weak_alias
-# ifdef SHARED
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
+#include <ifunc-strrchr.h>
+
+#if HAVE_STRRCHR_C
+# if HAVE_STRRCHR_IFUNC
+# define STRRCHR STRRCHR_C
+# undef weak_alias
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
__hidden_ver1 (__strrchr_c, __GI_strrchr, __strrchr_c);
-# endif /* SHARED */
+# endif
+# endif

# include <string/strrchr.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strrchr-vx.S b/sysdeps/s390/strrchr-vx.S
similarity index 94%
rename from sysdeps/s390/multiarch/strrchr-vx.S
rename to sysdeps/s390/strrchr-vx.S
index 8b3b989631..5f4ac14ee3 100644
--- a/sysdeps/s390/multiarch/strrchr-vx.S
+++ b/sysdeps/s390/strrchr-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strrchr.h>
+
+#if HAVE_STRRCHR_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -39,7 +41,7 @@
-v19=part of s with last occurence of c.
-v20=permute pattern
*/
-ENTRY(__strrchr_vx)
+ENTRY(STRRCHR_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -176,5 +178,15 @@ ENTRY(__strrchr_vx)
.Lpermute_mask:
.byte 0x0F,0x0E,0x0D,0x0C,0x0B,0x0A,0x09,0x08
.byte 0x07,0x06,0x05,0x04,0x03,0x02,0x01,0x00
-END(__strrchr_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRRCHR_Z13)
+
+# if ! HAVE_STRRCHR_IFUNC
+strong_alias (STRRCHR_Z13, strrchr)
+weak_alias (strrchr, rindex)
+# endif
+
+# if ! HAVE_STRRCHR_C && defined SHARED && IS_IN (libc)
+strong_alias (STRRCHR_Z13, __GI_strrchr)
+# endif
+
+#endif /* HAVE_STRRCHR_Z13 */
diff --git a/sysdeps/s390/multiarch/strrchr.c b/sysdeps/s390/strrchr.c
similarity index 66%
rename from sysdeps/s390/multiarch/strrchr.c
rename to sysdeps/s390/strrchr.c
index e00e25a3a4..9a8cecff0b 100644
--- a/sysdeps/s390/multiarch/strrchr.c
+++ b/sysdeps/s390/strrchr.c
@@ -16,15 +16,26 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strrchr.h>
+
+#if HAVE_STRRCHR_IFUNC
# define strrchr __redirect_strrchr
# include <string.h>
# undef strrchr
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strrchr, __strrchr, strrchr)
-weak_alias (strrchr, rindex);
+# if HAVE_STRRCHR_C
+extern __typeof (__redirect_strrchr) STRRCHR_C attribute_hidden;
+# endif
+
+# if HAVE_STRRCHR_Z13
+extern __typeof (__redirect_strrchr) STRRCHR_Z13 attribute_hidden;
+# endif

-#else
-# include <string/strrchr.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_strrchr, strrchr,
+ (HAVE_STRRCHR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRRCHR_Z13
+ : STRRCHR_DEFAULT
+ )
+weak_alias (strrchr, rindex)
+#endif /* HAVE_STRRCHR_IFUNC */

[Stefan Liebler]

The ifunc handling for wcschr is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.
Glibc internal calls will use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcschr variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcschr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcschr.
* sysdeps/s390/multiarch/wcschr-c.c: Move to ...
* sysdeps/s390/wcschr-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcschr-vx.S: Move to ...
* sysdeps/s390/wcschr-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcschr.c: Move to ...
* sysdeps/s390/wcschr.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcschr.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcschr.h | 53 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcschr-c.c | 39 ++++++++++-------
sysdeps/s390/{multiarch => }/wcschr-vx.S | 22 +++++++---
sysdeps/s390/{multiarch => }/wcschr.c | 23 +++++++---
7 files changed, 126 insertions(+), 32 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcschr.h
rename sysdeps/s390/{multiarch => }/wcschr-c.c (61%)
rename sysdeps/s390/{multiarch => }/wcschr-vx.S (88%)
rename sysdeps/s390/{multiarch => }/wcschr.c (71%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index e1253c04be..982e5abfb2 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -92,5 +92,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcscat wcscat-vx wcscat-c \
wcsncat wcsncat-vx wcsncat-c \
wcscmp wcscmp-vx wcscmp-c \
- wcsncmp wcsncmp-vx wcsncmp-c
+ wcsncmp wcsncmp-vx wcsncmp-c \
+ wcschr wcschr-vx wcschr-c
endif
diff --git a/sysdeps/s390/ifunc-wcschr.h b/sysdeps/s390/ifunc-wcschr.h
new file mode 100644
index 0000000000..2fe9110f7c
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcschr.h
@@ -0,0 +1,53 @@
+/* wcschr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSCHR_IFUNC 1
+#else
+# define HAVE_WCSCHR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSCHR_IFUNC_AND_VX_SUPPORT HAVE_WCSCHR_IFUNC
+#else
+# define HAVE_WCSCHR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSCHR_DEFAULT WCSCHR_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSCHR_C 1
+# define HAVE_WCSCHR_Z13 1
+#else
+# define WCSCHR_DEFAULT WCSCHR_C
+# define HAVE_WCSCHR_C 1
+# define HAVE_WCSCHR_Z13 HAVE_WCSCHR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSCHR_C
+# define WCSCHR_C __wcschr_c
+#else
+# define WCSCHR_C NULL
+#endif
+
+#if HAVE_WCSCHR_Z13
+# define WCSCHR_Z13 __wcschr_vx
+#else
+# define WCSCHR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index d1740fa8e4..b282613b47 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcschr wcschr-vx wcschr-c \
- wcschrnul wcschrnul-vx wcschrnul-c \
+sysdep_routines += wcschrnul wcschrnul-vx wcschrnul-c \
wcsrchr wcsrchr-vx wcsrchr-c \
wcsspn wcsspn-vx wcsspn-c \
wcspbrk wcspbrk-vx wcspbrk-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 8c9e788ae5..c69165465d 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -56,6 +56,7 @@
#include <ifunc-wcsncat.h>
#include <ifunc-wcscmp.h>
#include <ifunc-wcsncmp.h>
+#include <ifunc-wcschr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -554,6 +555,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSNCMP_IFUNC */

+#if HAVE_WCSCHR_IFUNC
+ IFUNC_IMPL (i, name, wcschr,
+# if HAVE_WCSCHR_Z13
+ IFUNC_IMPL_ADD (array, i, wcschr,
+ dl_hwcap & HWCAP_S390_VX, WCSCHR_Z13)
+# endif
+# if HAVE_WCSCHR_C
+ IFUNC_IMPL_ADD (array, i, wcschr, 1, WCSCHR_C)
+# endif
+ )
+#endif /* HAVE_WCSCHR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -562,8 +575,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcschr);
-
IFUNC_VX_IMPL (wcschrnul);

IFUNC_VX_IMPL (wcsrchr);
diff --git a/sysdeps/s390/multiarch/wcschr-c.c b/sysdeps/s390/wcschr-c.c
similarity index 61%
rename from sysdeps/s390/multiarch/wcschr-c.c
rename to sysdeps/s390/wcschr-c.c
index 8d6679c7f2..4908492a0a 100644
--- a/sysdeps/s390/multiarch/wcschr-c.c
+++ b/sysdeps/s390/wcschr-c.c
@@ -16,22 +16,29 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSCHR __wcschr_c
-
-# include <wchar.h>
-extern __typeof (__wcschr) __wcschr_c;
-# undef weak_alias
-# define weak_alias(name, alias)
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
- __hidden_ver1 (__wcschr_c, __GI_wcschr, __wcschr_c); \
- strong_alias (__wcschr_c, __wcschr_c_1); \
+#include <ifunc-wcschr.h>
+
+#if HAVE_WCSCHR_C
+# if HAVE_WCSCHR_IFUNC || HAVE_WCSCHR_Z13
+# define WCSCHR WCSCHR_C
+
+# undef weak_alias
+# define weak_alias(name, alias)
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_weak
+# define libc_hidden_weak(name)
+# undef libc_hidden_def
+# if ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wcschr_c, __GI_wcschr, __wcschr_c) __attribute__((weak)); \
+ strong_alias (__wcschr_c, __wcschr_c_1); \
__hidden_ver1 (__wcschr_c_1, __GI___wcschr, __wcschr_c_1);
-# undef libc_hidden_weak
-# define libc_hidden_weak(name)
-# endif /* SHARED */
+# else
+# define libc_hidden_def(name)
+# endif
+# endif
+# endif

# include <wcsmbs/wcschr.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/wcschr-vx.S b/sysdeps/s390/wcschr-vx.S
similarity index 88%
rename from sysdeps/s390/multiarch/wcschr-vx.S
rename to sysdeps/s390/wcschr-vx.S
index 94e5df7f36..dd24998390 100644
--- a/sysdeps/s390/multiarch/wcschr-vx.S
+++ b/sysdeps/s390/wcschr-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcschr.h>
+#if HAVE_WCSCHR_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -36,7 +37,7 @@
-v17=index of unequal
-v18=replicated c
*/
-ENTRY(__wcschr_vx)
+ENTRY(WCSCHR_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -98,6 +99,17 @@ ENTRY(__wcschr_vx)
lghi %r2,0 /* Return null if character not found. */
br %r14
.Lfallback:
- jg __wcschr_c
-END(__wcschr_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSCHR_C
+END(WCSCHR_Z13)
+
+# if ! HAVE_WCSCHR_IFUNC
+strong_alias (WCSCHR_Z13, __wcschr)
+weak_alias (__wcschr, wcschr)
+# endif
+
+# if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT \
+ && defined SHARED && IS_IN (libc)
+strong_alias (WCSCHR_Z13, __GI___wcschr)
+weak_alias (WCSCHR_Z13, __GI_wcschr)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcschr.c b/sysdeps/s390/wcschr.c
similarity index 71%
rename from sysdeps/s390/multiarch/wcschr.c
rename to sysdeps/s390/wcschr.c
index f44138f771..9923864ff8 100644
--- a/sysdeps/s390/multiarch/wcschr.c
+++ b/sysdeps/s390/wcschr.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcschr.h>
+
+#if HAVE_WCSCHR_IFUNC
# define wcschr __redirect_wcschr
# define __wcschr __redirect___wcschr
# include <wchar.h>
@@ -24,9 +26,18 @@
# undef __wcschr
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc_redirected (__redirect___wcschr, __wcschr)
-weak_alias (__wcschr, wcschr)
+# if HAVE_WCSCHR_C
+extern __typeof (__redirect___wcschr) WCSCHR_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcschr.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCSCHR_Z13
+extern __typeof (__redirect___wcschr) WCSCHR_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect___wcschr, __wcschr,
+ (HAVE_WCSCHR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSCHR_Z13
+ : WCSCHR_DEFAULT
+ )
+weak_alias (__wcschr, wcschr)
+#endif

[Stefan Liebler]

The ifunc handling for memccpy is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove memccpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add memccpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for memccpy.
* sysdeps/s390/multiarch/memccpy-c.c: Move to ...
* sysdeps/s390/memccpy-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/memccpy-vx.S: Move to ...
* sysdeps/s390/memccpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/memccpy.c: Move to ...
* sysdeps/s390/memccpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-memccpy.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-memccpy.h | 52 +++++++++++++++++++++++
sysdeps/s390/{multiarch => }/memccpy-c.c | 12 ++++--
sysdeps/s390/{multiarch => }/memccpy-vx.S | 16 +++++--
sysdeps/s390/{multiarch => }/memccpy.c | 23 +++++++---
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
7 files changed, 105 insertions(+), 19 deletions(-)
create mode 100644 sysdeps/s390/ifunc-memccpy.h
rename sysdeps/s390/{multiarch => }/memccpy-c.c (85%)
rename sysdeps/s390/{multiarch => }/memccpy-vx.S (95%)
rename sysdeps/s390/{multiarch => }/memccpy.c (68%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index a181eeac8a..649f29adc2 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -77,5 +77,6 @@ sysdep_routines += bzero memset memset-z900 \
strpbrk strpbrk-vx strpbrk-c \
strcspn strcspn-vx strcspn-c \
memchr memchr-vx memchr-z900 \
- rawmemchr rawmemchr-vx rawmemchr-c
+ rawmemchr rawmemchr-vx rawmemchr-c \
+ memccpy memccpy-vx memccpy-c
endif
diff --git a/sysdeps/s390/ifunc-memccpy.h b/sysdeps/s390/ifunc-memccpy.h
new file mode 100644
index 0000000000..8f7a1d0f9f
--- /dev/null
+++ b/sysdeps/s390/ifunc-memccpy.h
@@ -0,0 +1,52 @@
+/* memccpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_MEMCCPY_IFUNC 1
+#else
+# define HAVE_MEMCCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_MEMCCPY_IFUNC_AND_VX_SUPPORT HAVE_MEMCCPY_IFUNC
+#else
+# define HAVE_MEMCCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define MEMCCPY_DEFAULT MEMCCPY_Z13
+# define HAVE_MEMCCPY_C 0
+# define HAVE_MEMCCPY_Z13 1
+#else
+# define MEMCCPY_DEFAULT MEMCCPY_C
+# define HAVE_MEMCCPY_C 1
+# define HAVE_MEMCCPY_Z13 HAVE_MEMCCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_MEMCCPY_C
+# define MEMCCPY_C __memccpy_c
+#else
+# define MEMCCPY_C NULL
+#endif
+
+#if HAVE_MEMCCPY_Z13
+# define MEMCCPY_Z13 __memccpy_vx
+#else
+# define MEMCCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/memccpy-c.c b/sysdeps/s390/memccpy-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/memccpy-c.c
rename to sysdeps/s390/memccpy-c.c
index 1f4c548199..2b2f81eb9c 100644
--- a/sysdeps/s390/multiarch/memccpy-c.c
+++ b/sysdeps/s390/memccpy-c.c
@@ -16,10 +16,14 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define MEMCCPY __memccpy_c
+#include <ifunc-memccpy.h>
+
+#if HAVE_MEMCCPY_C
+# if HAVE_MEMCCPY_IFUNC
+# define MEMCCPY MEMCCPY_C
+# undef weak_alias
+# define weak_alias(a, b)
+#endif

-# include <string.h>
-extern __typeof (__memccpy) __memccpy_c;
# include <string/memccpy.c>
#endif
diff --git a/sysdeps/s390/multiarch/memccpy-vx.S b/sysdeps/s390/memccpy-vx.S
similarity index 95%
rename from sysdeps/s390/multiarch/memccpy-vx.S
rename to sysdeps/s390/memccpy-vx.S
index 150aa0e4a4..44f7bc5824 100644
--- a/sysdeps/s390/multiarch/memccpy-vx.S
+++ b/sysdeps/s390/memccpy-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-memccpy.h>
+
+#if HAVE_MEMCCPY_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -42,7 +44,7 @@
-v19=part #2 of s
-v31=save area for r6
*/
-ENTRY(__memccpy_vx)
+ENTRY(MEMCCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -152,5 +154,11 @@ ENTRY(__memccpy_vx)
vlgvg %r7,%v31,1
lghi %r2,0 /* Return null. */
br %r14
-END(__memccpy_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(MEMCCPY_Z13)
+
+# if ! HAVE_MEMCCPY_IFUNC
+strong_alias (MEMCCPY_Z13, __memccpy)
+weak_alias (__memccpy, memccpy)
+# endif
+
+#endif /* HAVE_MEMCCPY_Z13 */
diff --git a/sysdeps/s390/multiarch/memccpy.c b/sysdeps/s390/memccpy.c
similarity index 68%
rename from sysdeps/s390/multiarch/memccpy.c
rename to sysdeps/s390/memccpy.c
index 30aae82321..bcfeb31e86 100644
--- a/sysdeps/s390/multiarch/memccpy.c
+++ b/sysdeps/s390/memccpy.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-memccpy.h>
+
+#if HAVE_MEMCCPY_IFUNC
# include <string.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__memccpy)
-weak_alias (__memccpy, memccpy)
+# if HAVE_MEMCCPY_C
+extern __typeof (__memccpy) MEMCCPY_C attribute_hidden;
+# endif

-#else
-# include <string/memccpy.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_MEMCCPY_Z13
+extern __typeof (__memccpy) MEMCCPY_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__memccpy, __memccpy,
+ (HAVE_MEMCCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? MEMCCPY_Z13
+ : MEMCCPY_DEFAULT
+ )
+weak_alias (__memccpy, memccpy)
+#endif /* HAVE_MEMCCPY_IFUNC */
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index ac6cfcf9c7..d5a32fc309 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += memccpy memccpy-vx memccpy-c \
- memrchr memrchr-vx memrchr-c
+sysdep_routines += memrchr memrchr-vx memrchr-c
endif

ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index bf3b40e111..b8917747f0 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -44,6 +44,7 @@
#include <ifunc-strcspn.h>
#include <ifunc-memchr.h>
#include <ifunc-rawmemchr.h>
+#include <ifunc-memccpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -398,6 +399,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_RAWMEMCHR_IFUNC */

+#if HAVE_MEMCCPY_IFUNC
+ IFUNC_IMPL (i, name, memccpy,
+# if HAVE_MEMCCPY_Z13
+ IFUNC_IMPL_ADD (array, i, memccpy,
+ dl_hwcap & HWCAP_S390_VX, MEMCCPY_Z13)
+# endif
+# if HAVE_MEMCCPY_C
+ IFUNC_IMPL_ADD (array, i, memccpy, 1, MEMCCPY_C)
+# endif
+ )
+#endif /* HAVE_MEMCCPY_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -440,8 +453,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wmemchr);

- IFUNC_VX_IMPL (memccpy);
-
IFUNC_VX_IMPL (wmemset);

IFUNC_VX_IMPL (wmemcmp);

[Stefan Liebler]

The ifunc handling for strchrnul is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strchrnul variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strchrnul variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strchrnul.
* sysdeps/s390/multiarch/strchrnul-c.c: Move to ...
* sysdeps/s390/strchrnul-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strchrnul-vx.S: Move to ...
* sysdeps/s390/strchrnul-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strchrnul.c: Move to ...
* sysdeps/s390/strchrnul.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strchrnul.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strchrnul.h | 52 +++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 +++++-
sysdeps/s390/{multiarch => }/strchrnul-c.c | 14 ++++--
sysdeps/s390/{multiarch => }/strchrnul-vx.S | 16 +++++--
sysdeps/s390/{multiarch => }/strchrnul.c | 23 ++++++---
7 files changed, 106 insertions(+), 19 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strchrnul.h
rename sysdeps/s390/{multiarch => }/strchrnul-c.c (81%)
rename sysdeps/s390/{multiarch => }/strchrnul-vx.S (91%)
rename sysdeps/s390/{multiarch => }/strchrnul.c (67%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 59d6e845db..07ec5e969f 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -70,5 +70,6 @@ sysdep_routines += bzero memset memset-z900 \
strncat strncat-vx strncat-c \
strcmp strcmp-vx strcmp-z900 \
strncmp strncmp-vx strncmp-c \
- strchr strchr-vx strchr-c
+ strchr strchr-vx strchr-c \
+ strchrnul strchrnul-vx strchrnul-c
endif
diff --git a/sysdeps/s390/ifunc-strchrnul.h b/sysdeps/s390/ifunc-strchrnul.h
new file mode 100644
index 0000000000..cac817e6f0
--- /dev/null
+++ b/sysdeps/s390/ifunc-strchrnul.h
@@ -0,0 +1,52 @@
+/* strchrnul variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRCHRNUL_IFUNC 1
+#else
+# define HAVE_STRCHRNUL_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRCHRNUL_IFUNC_AND_VX_SUPPORT HAVE_STRCHRNUL_IFUNC
+#else
+# define HAVE_STRCHRNUL_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRCHRNUL_DEFAULT STRCHRNUL_Z13
+# define HAVE_STRCHRNUL_C 0
+# define HAVE_STRCHRNUL_Z13 1
+#else
+# define STRCHRNUL_DEFAULT STRCHRNUL_C
+# define HAVE_STRCHRNUL_C 1
+# define HAVE_STRCHRNUL_Z13 HAVE_STRCHRNUL_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRCHRNUL_C
+# define STRCHRNUL_C __strchrnul_c
+#else
+# define STRCHRNUL_C NULL
+#endif
+
+#if HAVE_STRCHRNUL_Z13
+# define STRCHRNUL_Z13 __strchrnul_vx
+#else
+# define STRCHRNUL_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index a8e9d0acd9..999a979fee 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strchrnul strchrnul-vx strchrnul-c \
- strrchr strrchr-vx strrchr-c \
+sysdep_routines += strrchr strrchr-vx strrchr-c \
strspn strspn-vx strspn-c \
strpbrk strpbrk-vx strpbrk-c \
strcspn strcspn-vx strcspn-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index e809ca3bac..0a47ffeac3 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -37,6 +37,7 @@
#include <ifunc-strcmp.h>
#include <ifunc-strncmp.h>
#include <ifunc-strchr.h>
+#include <ifunc-strchrnul.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -307,6 +308,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRCHR_IFUNC */

+#if HAVE_STRCHRNUL_IFUNC
+ IFUNC_IMPL (i, name, strchrnul,
+# if HAVE_STRCHRNUL_Z13
+ IFUNC_IMPL_ADD (array, i, strchrnul,
+ dl_hwcap & HWCAP_S390_VX, STRCHRNUL_Z13)
+# endif
+# if HAVE_STRCHRNUL_C
+ IFUNC_IMPL_ADD (array, i, strchrnul, 1, STRCHRNUL_C)
+# endif
+ )
+#endif /* HAVE_STRCHRNUL_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -337,7 +350,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcschr);

- IFUNC_VX_IMPL (strchrnul);
IFUNC_VX_IMPL (wcschrnul);

IFUNC_VX_IMPL (strrchr);
diff --git a/sysdeps/s390/multiarch/strchrnul-c.c b/sysdeps/s390/strchrnul-c.c
similarity index 81%
rename from sysdeps/s390/multiarch/strchrnul-c.c
rename to sysdeps/s390/strchrnul-c.c
index 020cebcf3e..585273f5de 100644
--- a/sysdeps/s390/multiarch/strchrnul-c.c
+++ b/sysdeps/s390/strchrnul-c.c
@@ -16,11 +16,15 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRCHRNUL __strchrnul_c
-# define __strchrnul STRCHRNUL
-# undef weak_alias
-# define weak_alias(name, alias)
+#include <ifunc-strchrnul.h>
+
+#if HAVE_STRCHRNUL_C
+# if HAVE_STRCHRNUL_IFUNC
+# define STRCHRNUL STRCHRNUL_C
+# define __strchrnul STRCHRNUL
+# undef weak_alias
+# define weak_alias(name, alias)
+# endif

# include <string/strchrnul.c>
#endif
diff --git a/sysdeps/s390/multiarch/strchrnul-vx.S b/sysdeps/s390/strchrnul-vx.S
similarity index 91%
rename from sysdeps/s390/multiarch/strchrnul-vx.S
rename to sysdeps/s390/strchrnul-vx.S
index d561825e04..0cd587bc32 100644
--- a/sysdeps/s390/multiarch/strchrnul-vx.S
+++ b/sysdeps/s390/strchrnul-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strchrnul.h>
+
+#if HAVE_STRCHRNUL_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -35,7 +37,7 @@
-v16=part of s
-v18=vector with c replicated in every byte
*/
-ENTRY(__strchrnul_vx)
+ENTRY(STRCHRNUL_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -89,5 +91,11 @@ ENTRY(__strchrnul_vx)

.Lend:
br %r14
-END(__strchrnul_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRCHRNUL_Z13)
+
+# if ! HAVE_STRCHRNUL_IFUNC
+strong_alias (STRCHRNUL_Z13, __strchrnul)
+weak_alias (__strchrnul, strchrnul)
+# endif
+
+#endif /* HAVE_STRCHRNUL_Z13 */
diff --git a/sysdeps/s390/multiarch/strchrnul.c b/sysdeps/s390/strchrnul.c
similarity index 67%
rename from sysdeps/s390/multiarch/strchrnul.c
rename to sysdeps/s390/strchrnul.c
index 62dfc6bd90..e9fefe1bdc 100644
--- a/sysdeps/s390/multiarch/strchrnul.c
+++ b/sysdeps/s390/strchrnul.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strchrnul.h>
+
+#if HAVE_STRCHRNUL_IFUNC
# include <string.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__strchrnul)
-weak_alias (__strchrnul, strchrnul)
+# if HAVE_STRCHRNUL_C
+extern __typeof (__strchrnul) STRCHRNUL_C attribute_hidden;
+# endif

-#else
-# include <string/strchrnul.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_STRCHRNUL_Z13
+extern __typeof (__strchrnul) STRCHRNUL_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__strchrnul, __strchrnul,
+ (HAVE_STRCHRNUL_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRCHRNUL_Z13
+ : STRCHRNUL_DEFAULT
+ )
+weak_alias (__strchrnul, strchrnul)
+#endif /* HAVE_STRCHRNUL_IFUNC */

[Stefan Liebler]

The ifunc handling for wcspbrk is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.
Glibc internal calls will use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcspbrk variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcspbrk variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcspbrk.
* sysdeps/s390/multiarch/wcspbrk-c.c: Move to ...
* sysdeps/s390/wcspbrk-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcspbrk-vx.S: Move to ...
* sysdeps/s390/wcspbrk-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcspbrk.c: Move to ...
* sysdeps/s390/wcspbrk.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcspbrk.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcspbrk.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcspbrk-c.c | 24 ++++++----
sysdeps/s390/{multiarch => }/wcspbrk-vx.S | 20 ++++++---
sysdeps/s390/{multiarch => }/wcspbrk.c | 20 ++++++---
7 files changed, 114 insertions(+), 24 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcspbrk.h
rename sysdeps/s390/{multiarch => }/wcspbrk-c.c (72%)
rename sysdeps/s390/{multiarch => }/wcspbrk-vx.S (97%)
rename sysdeps/s390/{multiarch => }/wcspbrk.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 2e48900c68..4decbf8f0a 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -96,5 +96,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcschr wcschr-vx wcschr-c \
wcschrnul wcschrnul-vx wcschrnul-c \
wcsrchr wcsrchr-vx wcsrchr-c \
- wcsspn wcsspn-vx wcsspn-c
+ wcsspn wcsspn-vx wcsspn-c \
+ wcspbrk wcspbrk-vx wcspbrk-c
endif
diff --git a/sysdeps/s390/ifunc-wcspbrk.h b/sysdeps/s390/ifunc-wcspbrk.h
new file mode 100644
index 0000000000..d3b9b7363a
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcspbrk.h
@@ -0,0 +1,53 @@
+/* wcspbrk variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSPBRK_IFUNC 1
+#else
+# define HAVE_WCSPBRK_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSPBRK_IFUNC_AND_VX_SUPPORT HAVE_WCSPBRK_IFUNC
+#else
+# define HAVE_WCSPBRK_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSPBRK_DEFAULT WCSPBRK_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSPBRK_C 1
+# define HAVE_WCSPBRK_Z13 1
+#else
+# define WCSPBRK_DEFAULT WCSPBRK_C
+# define HAVE_WCSPBRK_C 1
+# define HAVE_WCSPBRK_Z13 HAVE_WCSPBRK_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSPBRK_C
+# define WCSPBRK_C __wcspbrk_c
+#else
+# define WCSPBRK_C NULL
+#endif
+
+#if HAVE_WCSPBRK_Z13
+# define WCSPBRK_Z13 __wcspbrk_vx
+#else
+# define WCSPBRK_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 091f515084..e1e2d9dc74 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcspbrk wcspbrk-vx wcspbrk-c \
- wcscspn wcscspn-vx wcscspn-c \
+sysdep_routines += wcscspn wcscspn-vx wcscspn-c \
wmemchr wmemchr-vx wmemchr-c \
wmemset wmemset-vx wmemset-c \
wmemcmp wmemcmp-vx wmemcmp-c
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 6f4de2845b..89d6e8ad7e 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -60,6 +60,7 @@
#include <ifunc-wcschrnul.h>
#include <ifunc-wcsrchr.h>
#include <ifunc-wcsspn.h>
+#include <ifunc-wcspbrk.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -606,6 +607,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSSPN_IFUNC */

+#if HAVE_WCSPBRK_IFUNC
+ IFUNC_IMPL (i, name, wcspbrk,
+# if HAVE_WCSPBRK_Z13
+ IFUNC_IMPL_ADD (array, i, wcspbrk,
+ dl_hwcap & HWCAP_S390_VX, WCSPBRK_Z13)
+# endif
+# if HAVE_WCSPBRK_C
+ IFUNC_IMPL_ADD (array, i, wcspbrk, 1, WCSPBRK_C)
+# endif
+ )
+#endif /* HAVE_WCSPBRK_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -614,8 +627,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcspbrk);
-
IFUNC_VX_IMPL (wcscspn);

IFUNC_VX_IMPL (wmemchr);
diff --git a/sysdeps/s390/multiarch/wcspbrk-c.c b/sysdeps/s390/wcspbrk-c.c
similarity index 72%
rename from sysdeps/s390/multiarch/wcspbrk-c.c
rename to sysdeps/s390/wcspbrk-c.c
index 6b6e7aade4..e79c6a8550 100644
--- a/sysdeps/s390/multiarch/wcspbrk-c.c
+++ b/sysdeps/s390/wcspbrk-c.c
@@ -16,16 +16,22 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSPBRK __wcspbrk_c
+#include <ifunc-wcspbrk.h>

-# include <wchar.h>
-extern __typeof (wcspbrk) __wcspbrk_c;
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
+#if HAVE_WCSPBRK_C
+# if HAVE_WCSPBRK_IFUNC || HAVE_WCSPBRK_Z13
+# define WCSPBRK WCSPBRK_C
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# if ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define libc_hidden_def(name) \
__hidden_ver1 (__wcspbrk_c, __GI_wcspbrk, __wcspbrk_c);
-# endif /* SHARED */
+# else
+# define libc_hidden_def(name)
+# endif
+# endif
+# endif

# include <wcsmbs/wcspbrk.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/wcspbrk-vx.S b/sysdeps/s390/wcspbrk-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/wcspbrk-vx.S
rename to sysdeps/s390/wcspbrk-vx.S
index 5c89ec5d33..5870c4684d 100644
--- a/sysdeps/s390/multiarch/wcspbrk-vx.S
+++ b/sysdeps/s390/wcspbrk-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcspbrk.h>
+#if HAVE_WCSPBRK_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -59,7 +60,7 @@
otherwise =0;
r9: loaded byte count of vlbb accept-string
*/
-ENTRY(__wcspbrk_vx)
+ENTRY(WCSPBRK_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -310,6 +311,15 @@ ENTRY(__wcspbrk_vx)
lgr %r2,%r1
br %r14
.Lfallback:
- jg __wcspbrk_c
-END(__wcspbrk_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSPBRK_C
+END(WCSPBRK_Z13)
+
+# if ! HAVE_WCSPBRK_IFUNC
+strong_alias (WCSPBRK_Z13, wcspbrk)
+# endif
+
+# if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT \
+ && defined SHARED && IS_IN (libc)
+strong_alias (WCSPBRK_Z13, __GI_wcspbrk)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcspbrk.c b/sysdeps/s390/wcspbrk.c
similarity index 69%
rename from sysdeps/s390/multiarch/wcspbrk.c
rename to sysdeps/s390/wcspbrk.c
index 97876328b5..84ab911a24 100644
--- a/sysdeps/s390/multiarch/wcspbrk.c
+++ b/sysdeps/s390/wcspbrk.c
@@ -16,14 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcspbrk.h>
+
+#if HAVE_WCSPBRK_IFUNC
# define wcspbrk __redirect_wcspbrk
# include <wchar.h>
# undef wcspbrk
# include <ifunc-resolve.h>
+# if HAVE_WCSPBRK_C
+extern __typeof (__redirect_wcspbrk) WCSPBRK_C attribute_hidden;
+# endif

-s390_vx_libc_ifunc2_redirected (__redirect_wcspbrk, __wcspbrk, wcspbrk)
+# if HAVE_WCSPBRK_Z13
+extern __typeof (__redirect_wcspbrk) WCSPBRK_Z13 attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcspbrk.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_wcspbrk, wcspbrk,
+ (HAVE_WCSPBRK_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSPBRK_Z13
+ : WCSPBRK_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for rawmemchr is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove rawmemchr variants.
* sysdeps/s390/Makefile (sysdep_routines): Add rawmemchr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for rawmemchr.
* sysdeps/s390/multiarch/rawmemchr-c.c: Move to ...
* sysdeps/s390/rawmemchr-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/rawmemchr-vx.S: Move to ...
* sysdeps/s390/rawmemchr-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/rawmemchr.c: Move to ...
* sysdeps/s390/rawmemchr.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-rawmemchr.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-rawmemchr.h | 52 +++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 ++++-
.../{multiarch/rawmemchr.c => rawmemchr-c.c} | 25 +++++----
sysdeps/s390/{multiarch => }/rawmemchr-vx.S | 20 +++++--
.../{multiarch/rawmemchr-c.c => rawmemchr.c} | 33 +++++++-----
7 files changed, 118 insertions(+), 32 deletions(-)
create mode 100644 sysdeps/s390/ifunc-rawmemchr.h
rename sysdeps/s390/{multiarch/rawmemchr.c => rawmemchr-c.c} (67%)
rename sysdeps/s390/{multiarch => }/rawmemchr-vx.S (87%)
rename sysdeps/s390/{multiarch/rawmemchr-c.c => rawmemchr.c} (56%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 8f11ed5463..a181eeac8a 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -76,5 +76,6 @@ sysdep_routines += bzero memset memset-z900 \
strspn strspn-vx strspn-c \
strpbrk strpbrk-vx strpbrk-c \
strcspn strcspn-vx strcspn-c \
- memchr memchr-vx memchr-z900
+ memchr memchr-vx memchr-z900 \
+ rawmemchr rawmemchr-vx rawmemchr-c
endif
diff --git a/sysdeps/s390/ifunc-rawmemchr.h b/sysdeps/s390/ifunc-rawmemchr.h
new file mode 100644
index 0000000000..bfcbeae802
--- /dev/null
+++ b/sysdeps/s390/ifunc-rawmemchr.h
@@ -0,0 +1,52 @@
+/* rawmemchr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_RAWMEMCHR_IFUNC 1
+#else
+# define HAVE_RAWMEMCHR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_RAWMEMCHR_IFUNC_AND_VX_SUPPORT HAVE_RAWMEMCHR_IFUNC
+#else
+# define HAVE_RAWMEMCHR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define RAWMEMCHR_DEFAULT RAWMEMCHR_Z13
+# define HAVE_RAWMEMCHR_C 0
+# define HAVE_RAWMEMCHR_Z13 1
+#else
+# define RAWMEMCHR_DEFAULT RAWMEMCHR_C
+# define HAVE_RAWMEMCHR_C 1
+# define HAVE_RAWMEMCHR_Z13 HAVE_RAWMEMCHR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_RAWMEMCHR_C
+# define RAWMEMCHR_C __rawmemchr_c
+#else
+# define RAWMEMCHR_C NULL
+#endif
+
+#if HAVE_RAWMEMCHR_Z13
+# define RAWMEMCHR_Z13 __rawmemchr_vx
+#else
+# define RAWMEMCHR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index fa1f7b81db..ac6cfcf9c7 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += rawmemchr rawmemchr-vx rawmemchr-c \
- memccpy memccpy-vx memccpy-c \
+sysdep_routines += memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c
endif

diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index b4be014042..bf3b40e111 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -43,6 +43,7 @@
#include <ifunc-strpbrk.h>
#include <ifunc-strcspn.h>
#include <ifunc-memchr.h>
+#include <ifunc-rawmemchr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -385,6 +386,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_MEMCHR_IFUNC */

+#if HAVE_RAWMEMCHR_IFUNC
+ IFUNC_IMPL (i, name, rawmemchr,
+# if HAVE_RAWMEMCHR_Z13
+ IFUNC_IMPL_ADD (array, i, rawmemchr,
+ dl_hwcap & HWCAP_S390_VX, RAWMEMCHR_Z13)
+# endif
+# if HAVE_RAWMEMCHR_C
+ IFUNC_IMPL_ADD (array, i, rawmemchr, 1, RAWMEMCHR_C)
+# endif
+ )
+#endif /* HAVE_RAWMEMCHR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -426,7 +439,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
IFUNC_VX_IMPL (wcscspn);

IFUNC_VX_IMPL (wmemchr);
- IFUNC_VX_IMPL (rawmemchr);

IFUNC_VX_IMPL (memccpy);

diff --git a/sysdeps/s390/multiarch/rawmemchr.c b/sysdeps/s390/rawmemchr-c.c
similarity index 67%
rename from sysdeps/s390/multiarch/rawmemchr.c
rename to sysdeps/s390/rawmemchr-c.c
index 5fdb2252df..8b8208e542 100644
--- a/sysdeps/s390/multiarch/rawmemchr.c
+++ b/sysdeps/s390/rawmemchr-c.c
@@ -1,4 +1,4 @@
-/* Multiple versions of rawmemchr.
+/* Default rawmemchr implementation for S/390.
Copyright (C) 2015-2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.

@@ -16,16 +16,19 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define __rawmemchr __redirect___rawmemchr
-# include <string.h>
-# undef __rawmemchr
-# include <ifunc-resolve.h>
+#include <ifunc-rawmemchr.h>

-s390_vx_libc_ifunc2_redirected (__redirect___rawmemchr, __rawmemchr
- , __rawmemchr)
-weak_alias (__rawmemchr, rawmemchr)
+#if HAVE_RAWMEMCHR_C
+# if HAVE_RAWMEMCHR_IFUNC
+# define RAWMEMCHR RAWMEMCHR_C
+# undef weak_alias
+# define weak_alias(a, b)
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__rawmemchr_c, __GI___rawmemchr, __rawmemchr_c);
+# endif
+# endif

-#else
# include <string/rawmemchr.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+#endif
diff --git a/sysdeps/s390/multiarch/rawmemchr-vx.S b/sysdeps/s390/rawmemchr-vx.S
similarity index 87%
rename from sysdeps/s390/multiarch/rawmemchr-vx.S
rename to sysdeps/s390/rawmemchr-vx.S
index d5778be068..f04c0e8b61 100644
--- a/sysdeps/s390/multiarch/rawmemchr-vx.S
+++ b/sysdeps/s390/rawmemchr-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-rawmemchr.h>
+
+#if HAVE_RAWMEMCHR_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -37,7 +39,7 @@
-v17=index of unequal
-v18=c replicated
*/
-ENTRY(__rawmemchr_vx)
+ENTRY(RAWMEMCHR_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -88,5 +90,15 @@ ENTRY(__rawmemchr_vx)
.Lend_found:
la %r2,0(%r5,%r2) /* Return pointer to character. */
br %r14
-END(__rawmemchr_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(RAWMEMCHR_Z13)
+
+# if ! HAVE_RAWMEMCHR_IFUNC
+strong_alias (RAWMEMCHR_Z13, __rawmemchr)
+weak_alias (__rawmemchr, rawmemchr)
+# endif
+
+# if ! HAVE_RAWMEMCHR_C && defined SHARED && IS_IN (libc)
+strong_alias (RAWMEMCHR_Z13, __GI___rawmemchr)
+# endif
+
+#endif /* HAVE_RAWMEMCHR_Z13 */
diff --git a/sysdeps/s390/multiarch/rawmemchr-c.c b/sysdeps/s390/rawmemchr.c
similarity index 56%
rename from sysdeps/s390/multiarch/rawmemchr-c.c
rename to sysdeps/s390/rawmemchr.c
index f43c883a76..d9263ce208 100644
--- a/sysdeps/s390/multiarch/rawmemchr-c.c
+++ b/sysdeps/s390/rawmemchr.c
@@ -1,4 +1,4 @@
-/* Default rawmemchr implementation for S/390.
+/* Multiple versions of rawmemchr.
Copyright (C) 2015-2018 Free Software Foundation, Inc.
This file is part of the GNU C Library.

@@ -16,19 +16,26 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-rawmemchr.h>
+
+#if HAVE_RAWMEMCHR_IFUNC
+# define __rawmemchr __redirect___rawmemchr
# include <string.h>
+# undef __rawmemchr
+# include <ifunc-resolve.h>

-# define RAWMEMCHR __rawmemchr_c
-# undef weak_alias
-# define weak_alias(a, b)
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
- __hidden_ver1 (__rawmemchr_c, __GI___rawmemchr, __rawmemchr_c);
-# endif /* SHARED */
+# if HAVE_RAWMEMCHR_C
+extern __typeof (__redirect___rawmemchr) RAWMEMCHR_C attribute_hidden;
+# endif

-extern __typeof (rawmemchr) __rawmemchr_c attribute_hidden;
+# if HAVE_RAWMEMCHR_Z13
+extern __typeof (__redirect___rawmemchr) RAWMEMCHR_Z13 attribute_hidden;
+# endif

-# include <string/rawmemchr.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect___rawmemchr, __rawmemchr,
+ (HAVE_RAWMEMCHR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? RAWMEMCHR_Z13
+ : RAWMEMCHR_DEFAULT
+ )
+weak_alias (__rawmemchr, rawmemchr)
+#endif /* HAVE_RAWMEMCHR_IFUNC */

[Stefan Liebler]

The ifunc handling for wcslen is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcslen variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcslen variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcslen.
* sysdeps/s390/multiarch/wcslen-c.c: Move to ...
* sysdeps/s390/wcslen-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcslen-vx.S: Move to ...
* sysdeps/s390/wcslen-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcslen.c: Move to ...
* sysdeps/s390/wcslen.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcslen.h: New file.
---
sysdeps/s390/Makefile | 4 ++
sysdeps/s390/ifunc-wcslen.h | 53 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcslen-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcslen-vx.S | 16 ++++---
sysdeps/s390/{multiarch => }/wcslen.c | 23 +++++++---
7 files changed, 105 insertions(+), 19 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcslen.h
rename sysdeps/s390/{multiarch => }/wcslen-c.c (86%)
rename sysdeps/s390/{multiarch => }/wcslen-vx.S (92%)
rename sysdeps/s390/{multiarch => }/wcslen.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 0e86242c5d..10c0e1bea0 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -81,3 +81,7 @@ sysdep_routines += bzero memset memset-z900 \
memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c
endif
+
+ifeq ($(subdir),wcsmbs)
+sysdep_routines += wcslen wcslen-vx wcslen-c
+endif
diff --git a/sysdeps/s390/ifunc-wcslen.h b/sysdeps/s390/ifunc-wcslen.h
new file mode 100644
index 0000000000..50d879caf2
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcslen.h
@@ -0,0 +1,53 @@
+/* wcslen variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSLEN_IFUNC 1
+#else
+# define HAVE_WCSLEN_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSLEN_IFUNC_AND_VX_SUPPORT HAVE_WCSLEN_IFUNC
+#else
+# define HAVE_WCSLEN_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSLEN_DEFAULT WCSLEN_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSLEN_C 1
+# define HAVE_WCSLEN_Z13 1
+#else
+# define WCSLEN_DEFAULT WCSLEN_C
+# define HAVE_WCSLEN_C 1
+# define HAVE_WCSLEN_Z13 HAVE_WCSLEN_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSLEN_C
+# define WCSLEN_C __wcslen_c
+#else
+# define WCSLEN_C NULL
+#endif
+
+#if HAVE_WCSLEN_Z13
+# define WCSLEN_Z13 __wcslen_vx
+#else
+# define WCSLEN_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 260b514936..421d40d020 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcslen wcslen-vx wcslen-c \
- wcsnlen wcsnlen-vx wcsnlen-c \
+sysdep_routines += wcsnlen wcsnlen-vx wcsnlen-c \
wcscpy wcscpy-vx wcscpy-c \
wcpcpy wcpcpy-vx wcpcpy-c \
wcsncpy wcsncpy-vx wcsncpy-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 0f01b99691..7bf5f14c01 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -46,6 +46,7 @@
#include <ifunc-rawmemchr.h>
#include <ifunc-memccpy.h>
#include <ifunc-memrchr.h>
+#include <ifunc-wcslen.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -424,6 +425,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_MEMRCHR_IFUNC */

+#if HAVE_WCSLEN_IFUNC
+ IFUNC_IMPL (i, name, wcslen,
+# if HAVE_WCSLEN_Z13
+ IFUNC_IMPL_ADD (array, i, wcslen,
+ dl_hwcap & HWCAP_S390_VX, WCSLEN_Z13)
+# endif
+# if HAVE_WCSLEN_C
+ IFUNC_IMPL_ADD (array, i, wcslen, 1, WCSLEN_C)
+# endif
+ )
+#endif /* HAVE_WCSLEN_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -432,8 +445,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcslen);
-
IFUNC_VX_IMPL (wcsnlen);

IFUNC_VX_IMPL (wcscpy);
diff --git a/sysdeps/s390/multiarch/wcslen-c.c b/sysdeps/s390/wcslen-c.c
similarity index 86%
rename from sysdeps/s390/multiarch/wcslen-c.c
rename to sysdeps/s390/wcslen-c.c
index 32a23e206d..45399cff3a 100644
--- a/sysdeps/s390/multiarch/wcslen-c.c
+++ b/sysdeps/s390/wcslen-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSLEN __wcslen_c
+#include <ifunc-wcslen.h>
+
+#if HAVE_WCSLEN_C
+# if HAVE_WCSLEN_IFUNC || HAVE_WCSLEN_Z13
+# define WCSLEN WCSLEN_C
+# endif

-# include <wchar.h>
-extern __typeof (__wcslen) __wcslen_c;
# include <wcsmbs/wcslen.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcslen-vx.S b/sysdeps/s390/wcslen-vx.S
similarity index 92%
rename from sysdeps/s390/multiarch/wcslen-vx.S
rename to sysdeps/s390/wcslen-vx.S
index 337cbed6ec..114f7ef743 100644
--- a/sysdeps/s390/multiarch/wcslen-vx.S
+++ b/sysdeps/s390/wcslen-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcslen.h>
+#if HAVE_WCSLEN_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -34,7 +35,7 @@
-r5=current_len and return_value
-v16=part of s
*/
-ENTRY(__wcslen_vx)
+ENTRY(WCSLEN_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -86,6 +87,11 @@ ENTRY(__wcslen_vx)
srlg %r2,%r2,2 /* Convert byte-count to character-count. */
br %r14
.Lfallback:
- jg __wcslen_c
-END(__wcslen_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSLEN_C
+END(WCSLEN_Z13)
+
+# if ! HAVE_WCSLEN_IFUNC
+strong_alias (WCSLEN_Z13, __wcslen)
+weak_alias (__wcslen, wcslen)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcslen.c b/sysdeps/s390/wcslen.c
similarity index 70%
rename from sysdeps/s390/multiarch/wcslen.c
rename to sysdeps/s390/wcslen.c
index 3a1d1a32c9..a5eee83f6c 100644
--- a/sysdeps/s390/multiarch/wcslen.c
+++ b/sysdeps/s390/wcslen.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcslen.h>
+
+#if HAVE_WCSLEN_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__wcslen)
-weak_alias (__wcslen, wcslen)
+# if HAVE_WCSLEN_C
+extern __typeof (__wcslen) WCSLEN_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcslen.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCSLEN_Z13
+extern __typeof (__wcslen) WCSLEN_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__wcslen, __wcslen,
+ (HAVE_WCSLEN_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSLEN_Z13
+ : WCSLEN_DEFAULT
+ )
+weak_alias (__wcslen, wcslen)
+#endif

[Stefan Liebler]

The ifunc handling for strncmp is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strncmp variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strncmp variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strncmp.
* sysdeps/s390/multiarch/strncmp-c.c: Move to ...
* sysdeps/s390/strncmp-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strncmp-vx.S: Move to ...
* sysdeps/s390/strncmp-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strncmp.c: Move to ...
* sysdeps/s390/strncmp.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strncmp.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strncmp.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 +++++-
sysdeps/s390/{multiarch => }/strncmp-c.c | 18 +++++---
sysdeps/s390/{multiarch => }/strncmp-vx.S | 19 +++++++--
sysdeps/s390/{multiarch => }/strncmp.c | 21 ++++++---
7 files changed, 110 insertions(+), 20 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strncmp.h
rename sysdeps/s390/{multiarch => }/strncmp-c.c (78%)
rename sysdeps/s390/{multiarch => }/strncmp-vx.S (93%)
rename sysdeps/s390/{multiarch => }/strncmp.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 9e7d5625c9..3aaf5a0d40 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -68,5 +68,6 @@ sysdep_routines += bzero memset memset-z900 \
stpncpy stpncpy-vx stpncpy-c \
strcat strcat-vx strcat-c \
strncat strncat-vx strncat-c \
- strcmp strcmp-vx strcmp-z900
+ strcmp strcmp-vx strcmp-z900 \
+ strncmp strncmp-vx strncmp-c
endif
diff --git a/sysdeps/s390/ifunc-strncmp.h b/sysdeps/s390/ifunc-strncmp.h
new file mode 100644
index 0000000000..511b3e9720
--- /dev/null
+++ b/sysdeps/s390/ifunc-strncmp.h
@@ -0,0 +1,52 @@
+/* strncmp variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRNCMP_IFUNC 1
+#else
+# define HAVE_STRNCMP_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRNCMP_IFUNC_AND_VX_SUPPORT HAVE_STRNCMP_IFUNC
+#else
+# define HAVE_STRNCMP_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRNCMP_DEFAULT STRNCMP_Z13
+# define HAVE_STRNCMP_C 0
+# define HAVE_STRNCMP_Z13 1
+#else
+# define STRNCMP_DEFAULT STRNCMP_C
+# define HAVE_STRNCMP_C 1
+# define HAVE_STRNCMP_Z13 HAVE_STRNCMP_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRNCMP_C
+# define STRNCMP_C __strncmp_c
+#else
+# define STRNCMP_C NULL
+#endif
+
+#if HAVE_STRNCMP_Z13
+# define STRNCMP_Z13 __strncmp_vx
+#else
+# define STRNCMP_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 97421a4996..381376bf9f 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strncmp strncmp-vx strncmp-c \
- strchr strchr-vx strchr-c \
+sysdep_routines += strchr strchr-vx strchr-c \
strchrnul strchrnul-vx strchrnul-c \
strrchr strrchr-vx strrchr-c \
strspn strspn-vx strspn-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 44637c431b..d982de5788 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -35,6 +35,7 @@
#include <ifunc-strcat.h>
#include <ifunc-strncat.h>
#include <ifunc-strcmp.h>
+#include <ifunc-strncmp.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -281,6 +282,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRCMP_IFUNC */

+#if HAVE_STRNCMP_IFUNC
+ IFUNC_IMPL (i, name, strncmp,
+# if HAVE_STRNCMP_Z13
+ IFUNC_IMPL_ADD (array, i, strncmp,
+ dl_hwcap & HWCAP_S390_VX, STRNCMP_Z13)
+# endif
+# if HAVE_STRNCMP_C
+ IFUNC_IMPL_ADD (array, i, strncmp, 1, STRNCMP_C)
+# endif
+ )
+#endif /* HAVE_STRNCMP_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -307,7 +320,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcscmp);

- IFUNC_VX_IMPL (strncmp);
IFUNC_VX_IMPL (wcsncmp);

IFUNC_VX_IMPL (strchr);
diff --git a/sysdeps/s390/multiarch/strncmp-c.c b/sysdeps/s390/strncmp-c.c
similarity index 78%
rename from sysdeps/s390/multiarch/strncmp-c.c
rename to sysdeps/s390/strncmp-c.c
index e54277ec1b..c7ffdb03e3 100644
--- a/sysdeps/s390/multiarch/strncmp-c.c
+++ b/sysdeps/s390/strncmp-c.c
@@ -16,13 +16,17 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRNCMP __strncmp_c
-# ifdef SHARED
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
+#include <ifunc-strncmp.h>
+
+#if HAVE_STRNCMP_C
+# if HAVE_STRNCMP_IFUNC
+# define STRNCMP STRNCMP_C
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
__hidden_ver1 (__strncmp_c, __GI_strncmp, __strncmp_c);
-# endif /* SHARED */
+# endif
+# endif

# include <string/strncmp.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strncmp-vx.S b/sysdeps/s390/strncmp-vx.S
similarity index 93%
rename from sysdeps/s390/multiarch/strncmp-vx.S
rename to sysdeps/s390/strncmp-vx.S
index 168fd657da..f557afb336 100644
--- a/sysdeps/s390/multiarch/strncmp-vx.S
+++ b/sysdeps/s390/strncmp-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strncmp.h>
+
+#if HAVE_STRNCMP_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -37,7 +39,7 @@
-v17=part of s2
-v18=index of unequal
*/
-ENTRY(__strncmp_vx)
+ENTRY(STRNCMP_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -133,5 +135,14 @@ ENTRY(__strncmp_vx)
.Lend_equal:
lghi %r2,0
br %r14
-END(__strncmp_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRNCMP_Z13)
+
+# if ! HAVE_STRNCMP_IFUNC
+strong_alias (STRNCMP_Z13, strncmp)
+# endif
+
+# if ! HAVE_STRNCMP_C && defined SHARED && IS_IN (libc)
+strong_alias (STRNCMP_Z13, __GI_strncmp)
+# endif
+
+#endif /* HAVE_STRNCMP_Z13 */
diff --git a/sysdeps/s390/multiarch/strncmp.c b/sysdeps/s390/strncmp.c
similarity index 70%
rename from sysdeps/s390/multiarch/strncmp.c
rename to sysdeps/s390/strncmp.c
index 0ec472c3b0..71351273c4 100644
--- a/sysdeps/s390/multiarch/strncmp.c
+++ b/sysdeps/s390/strncmp.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strncmp.h>
+
+#if HAVE_STRNCMP_IFUNC
# define strncmp __redirect_strncmp
/* Omit the strncmp inline definitions because it would redefine strncmp. */
# define __NO_STRING_INLINES
@@ -24,8 +26,17 @@
# undef strncmp
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strncmp, __strncmp, strncmp)
+# if HAVE_STRNCMP_C
+extern __typeof (__redirect_strncmp) STRNCMP_C attribute_hidden;
+# endif
+
+# if HAVE_STRNCMP_Z13
+extern __typeof (__redirect_strncmp) STRNCMP_Z13 attribute_hidden;
+# endif

-#else
-# include <string/strncmp.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_strncmp, strncmp,
+ (HAVE_STRNCMP_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRNCMP_Z13
+ : STRNCMP_DEFAULT
+ )
+#endif /* HAVE_STRNCMP_IFUNC */

[Stefan Liebler]

The ifunc handling for wcschrnul is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcschrnul variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcschrnul variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcschrnul.
* sysdeps/s390/multiarch/wcschrnul-c.c: Move to ...
* sysdeps/s390/wcschrnul-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcschrnul-vx.S: Move to ...
* sysdeps/s390/wcschrnul-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcschrnul.c: Move to ...
* sysdeps/s390/wcschrnul.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcschrnul.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcschrnul.h | 53 +++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 +++++-
sysdeps/s390/{multiarch => }/wcschrnul-c.c | 10 ++--
sysdeps/s390/{multiarch => }/wcschrnul-vx.S | 16 +++++--
sysdeps/s390/{multiarch => }/wcschrnul.c | 23 ++++++---
7 files changed, 103 insertions(+), 20 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcschrnul.h
rename sysdeps/s390/{multiarch => }/wcschrnul-c.c (85%)
rename sysdeps/s390/{multiarch => }/wcschrnul-vx.S (92%)
rename sysdeps/s390/{multiarch => }/wcschrnul.c (68%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 982e5abfb2..a8cd264521 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -93,5 +93,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsncat wcsncat-vx wcsncat-c \
wcscmp wcscmp-vx wcscmp-c \
wcsncmp wcsncmp-vx wcsncmp-c \
- wcschr wcschr-vx wcschr-c
+ wcschr wcschr-vx wcschr-c \
+ wcschrnul wcschrnul-vx wcschrnul-c
endif
diff --git a/sysdeps/s390/ifunc-wcschrnul.h b/sysdeps/s390/ifunc-wcschrnul.h
new file mode 100644
index 0000000000..39d4120eb3
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcschrnul.h
@@ -0,0 +1,53 @@
+/* wcschrnul variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSCHRNUL_IFUNC 1
+#else
+# define HAVE_WCSCHRNUL_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSCHRNUL_IFUNC_AND_VX_SUPPORT HAVE_WCSCHRNUL_IFUNC
+#else
+# define HAVE_WCSCHRNUL_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSCHRNUL_DEFAULT WCSCHRNUL_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSCHRNUL_C 1
+# define HAVE_WCSCHRNUL_Z13 1
+#else
+# define WCSCHRNUL_DEFAULT WCSCHRNUL_C
+# define HAVE_WCSCHRNUL_C 1
+# define HAVE_WCSCHRNUL_Z13 HAVE_WCSCHRNUL_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSCHRNUL_C
+# define WCSCHRNUL_C __wcschrnul_c
+#else
+# define WCSCHRNUL_C NULL
+#endif
+
+#if HAVE_WCSCHRNUL_Z13
+# define WCSCHRNUL_Z13 __wcschrnul_vx
+#else
+# define WCSCHRNUL_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index b282613b47..0eb01d5d84 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcschrnul wcschrnul-vx wcschrnul-c \
- wcsrchr wcsrchr-vx wcsrchr-c \
+sysdep_routines += wcsrchr wcsrchr-vx wcsrchr-c \
wcsspn wcsspn-vx wcsspn-c \
wcspbrk wcspbrk-vx wcspbrk-c \
wcscspn wcscspn-vx wcscspn-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index c69165465d..553a76c39e 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -57,6 +57,7 @@
#include <ifunc-wcscmp.h>
#include <ifunc-wcsncmp.h>
#include <ifunc-wcschr.h>
+#include <ifunc-wcschrnul.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -567,6 +568,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSCHR_IFUNC */

+#if HAVE_WCSCHRNUL_IFUNC
+ IFUNC_IMPL (i, name, wcschrnul,
+# if HAVE_WCSCHRNUL_Z13
+ IFUNC_IMPL_ADD (array, i, wcschrnul,
+ dl_hwcap & HWCAP_S390_VX, WCSCHRNUL_Z13)
+# endif
+# if HAVE_WCSCHRNUL_C
+ IFUNC_IMPL_ADD (array, i, wcschrnul, 1, WCSCHRNUL_C)
+# endif
+ )
+#endif /* HAVE_WCSCHRNUL_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -575,8 +588,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcschrnul);
-
IFUNC_VX_IMPL (wcsrchr);

IFUNC_VX_IMPL (wcsspn);
diff --git a/sysdeps/s390/multiarch/wcschrnul-c.c b/sysdeps/s390/wcschrnul-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/wcschrnul-c.c
rename to sysdeps/s390/wcschrnul-c.c
index 00e776a3ad..410cee1399 100644
--- a/sysdeps/s390/multiarch/wcschrnul-c.c
+++ b/sysdeps/s390/wcschrnul-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSCHRNUL __wcschrnul_c
+#include <ifunc-wcschrnul.h>
+
+#if HAVE_WCSCHRNUL_C
+# if HAVE_WCSCHRNUL_IFUNC || HAVE_WCSCHRNUL_Z13
+# define WCSCHRNUL WCSCHRNUL_C
+# endif

-# include <wchar.h>
-extern __typeof (__wcschrnul) __wcschrnul_c;
# include <wcsmbs/wcschrnul.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcschrnul-vx.S b/sysdeps/s390/wcschrnul-vx.S
similarity index 92%
rename from sysdeps/s390/multiarch/wcschrnul-vx.S
rename to sysdeps/s390/wcschrnul-vx.S
index ebcd32b870..4ffb937c6b 100644
--- a/sysdeps/s390/multiarch/wcschrnul-vx.S
+++ b/sysdeps/s390/wcschrnul-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcschrnul.h>
+#if HAVE_WCSCHRNUL_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -35,7 +36,7 @@
-v16=part of s
-v18=vector with c replicated in every byte
*/
-ENTRY(__wcschrnul_vx)
+ENTRY(WCSCHRNUL_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -92,6 +93,11 @@ ENTRY(__wcschrnul_vx)
.Lend:
br %r14
.Lfallback:
- jg __wcschrnul_c
-END(__wcschrnul_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSCHRNUL_C
+END(WCSCHRNUL_Z13)
+
+# if ! HAVE_WCSCHRNUL_IFUNC
+strong_alias (WCSCHRNUL_Z13, __wcschrnul)
+weak_alias (__wcschrnul, wcschrnul)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcschrnul.c b/sysdeps/s390/wcschrnul.c
similarity index 68%
rename from sysdeps/s390/multiarch/wcschrnul.c
rename to sysdeps/s390/wcschrnul.c
index 807d7ee089..ab5aaf0402 100644
--- a/sysdeps/s390/multiarch/wcschrnul.c
+++ b/sysdeps/s390/wcschrnul.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcschrnul.h>
+
+#if HAVE_WCSCHRNUL_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__wcschrnul)
-weak_alias (__wcschrnul, wcschrnul)
+# if HAVE_WCSCHRNUL_C
+extern __typeof (__wcschrnul) WCSCHRNUL_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcschrnul.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCSCHRNUL_Z13
+extern __typeof (__wcschrnul) WCSCHRNUL_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__wcschrnul, __wcschrnul,
+ (HAVE_WCSCHRNUL_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSCHRNUL_Z13
+ : WCSCHRNUL_DEFAULT
+ )
+weak_alias (__wcschrnul, wcschrnul)
+#endif

[Stefan Liebler]

The ifunc handling for strncat is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strncat variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strncat variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strncat.
* sysdeps/s390/multiarch/strncat-c.c: Move to ...
* sysdeps/s390/strncat-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strncat-vx.S: Move to ...
* sysdeps/s390/strncat-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strncat.c: Move to ...
* sysdeps/s390/strncat.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strncat.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strncat.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 +++++-
sysdeps/s390/{multiarch => }/strncat-c.c | 10 +++--
sysdeps/s390/{multiarch => }/strncat-vx.S | 21 +++++++--
sysdeps/s390/{multiarch => }/strncat.c | 21 ++++++---
7 files changed, 108 insertions(+), 16 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strncat.h
rename sysdeps/s390/{multiarch => }/strncat-c.c (86%)
rename sysdeps/s390/{multiarch => }/strncat-vx.S (94%)
rename sysdeps/s390/{multiarch => }/strncat.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 3783f44252..ec0fb8c2ad 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -66,5 +66,6 @@ sysdep_routines += bzero memset memset-z900 \
stpcpy stpcpy-vx stpcpy-c \
strncpy strncpy-vx strncpy-z900 \
stpncpy stpncpy-vx stpncpy-c \
- strcat strcat-vx strcat-c
+ strcat strcat-vx strcat-c \
+ strncat strncat-vx strncat-c
endif
diff --git a/sysdeps/s390/ifunc-strncat.h b/sysdeps/s390/ifunc-strncat.h
new file mode 100644
index 0000000000..bb164dcc32
--- /dev/null
+++ b/sysdeps/s390/ifunc-strncat.h
@@ -0,0 +1,52 @@
+/* strncat variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRNCAT_IFUNC 1
+#else
+# define HAVE_STRNCAT_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRNCAT_IFUNC_AND_VX_SUPPORT HAVE_STRNCAT_IFUNC
+#else
+# define HAVE_STRNCAT_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRNCAT_DEFAULT STRNCAT_Z13
+# define HAVE_STRNCAT_C 0
+# define HAVE_STRNCAT_Z13 1
+#else
+# define STRNCAT_DEFAULT STRNCAT_C
+# define HAVE_STRNCAT_C 1
+# define HAVE_STRNCAT_Z13 HAVE_STRNCAT_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRNCAT_C
+# define STRNCAT_C __strncat_c
+#else
+# define STRNCAT_C NULL
+#endif
+
+#if HAVE_STRNCAT_Z13
+# define STRNCAT_Z13 __strncat_vx
+#else
+# define STRNCAT_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 9b66237aaf..24be3eac51 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strncat strncat-vx strncat-c \
- strcmp strcmp-vx \
+sysdep_routines += strcmp strcmp-vx \
strncmp strncmp-vx strncmp-c \
strchr strchr-vx strchr-c \
strchrnul strchrnul-vx strchrnul-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 1b7f3df3a3..3abcaf08e0 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -33,6 +33,7 @@
#include <ifunc-strncpy.h>
#include <ifunc-stpncpy.h>
#include <ifunc-strcat.h>
+#include <ifunc-strncat.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -255,6 +256,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRCAT_IFUNC */

+#if HAVE_STRNCAT_IFUNC
+ IFUNC_IMPL (i, name, strncat,
+# if HAVE_STRNCAT_Z13
+ IFUNC_IMPL_ADD (array, i, strncat,
+ dl_hwcap & HWCAP_S390_VX, STRNCAT_Z13)
+# endif
+# if HAVE_STRNCAT_C
+ IFUNC_IMPL_ADD (array, i, strncat, 1, STRNCAT_C)
+# endif
+ )
+#endif /* HAVE_STRNCAT_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -277,7 +290,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcscat);

- IFUNC_VX_IMPL (strncat);
IFUNC_VX_IMPL (wcsncat);

IFUNC_VX_IMPL (strcmp);
diff --git a/sysdeps/s390/multiarch/strncat-c.c b/sysdeps/s390/strncat-c.c
similarity index 86%
rename from sysdeps/s390/multiarch/strncat-c.c
rename to sysdeps/s390/strncat-c.c
index 9e6c245ccb..86df89887c 100644
--- a/sysdeps/s390/multiarch/strncat-c.c
+++ b/sysdeps/s390/strncat-c.c
@@ -16,8 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRNCAT __strncat_c
-# define STRNCAT_PRIMARY
+#include <ifunc-strncat.h>
+
+#if HAVE_STRNCAT_C
+# if HAVE_STRNCAT_IFUNC
+# define STRNCAT STRNCAT_C
+# define STRNCAT_PRIMARY
+# endif
# include <string/strncat.c>
#endif
diff --git a/sysdeps/s390/multiarch/strncat-vx.S b/sysdeps/s390/strncat-vx.S
similarity index 94%
rename from sysdeps/s390/multiarch/strncat-vx.S
rename to sysdeps/s390/strncat-vx.S
index e6584d0f43..76345e7dd7 100644
--- a/sysdeps/s390/multiarch/strncat-vx.S
+++ b/sysdeps/s390/strncat-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strncat.h>
+#if HAVE_STRNCAT_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -40,7 +41,7 @@
-v18=part of src
-v31=register save area for r6, r7
*/
-ENTRY(__strncat_vx)
+ENTRY(STRNCAT_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -235,5 +236,17 @@ ENTRY(__strncat_vx)

vl %v16,0(%r5,%r3) /* Load s. */
j .Lcpy_lt64
-END(__strncat_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRNCAT_Z13)
+
+# if ! HAVE_STRNCAT_IFUNC
+strong_alias (STRNCAT_Z13, strncat)
+# endif
+
+# if ! HAVE_STRNCAT_C
+/* See string/strncat.c and define STRNCAT_PRIMARY. */
+strong_alias (STRNCAT_Z13, __strncat)
+# if defined SHARED && IS_IN (libc)
+strong_alias (__strncat, __GI___strncat)
+# endif
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/strncat.c b/sysdeps/s390/strncat.c
similarity index 69%
rename from sysdeps/s390/multiarch/strncat.c
rename to sysdeps/s390/strncat.c
index 94b8dffa85..b4b3656c5a 100644
--- a/sysdeps/s390/multiarch/strncat.c
+++ b/sysdeps/s390/strncat.c
@@ -16,12 +16,23 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strncat.h>
+
+#if HAVE_STRNCAT_IFUNC
# include <string.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2 (__strncat, strncat)
+# if HAVE_STRNCAT_C
+extern __typeof (__strncat) STRNCAT_C attribute_hidden;
+# endif
+
+# if HAVE_STRNCAT_Z13
+extern __typeof (__strncat) STRNCAT_Z13 attribute_hidden;
+# endif

-#else
-# include <string/strncat.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__strncat, strncat,
+ (HAVE_STRNCAT_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRNCAT_Z13
+ : STRNCAT_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for strcat is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strcat variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strcat variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strcat.
* sysdeps/s390/multiarch/strcat-c.c: Move to ...
* sysdeps/s390/strcat-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strcat-vx.S: Move to ...
* sysdeps/s390/strcat-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strcat.c: Move to ...
* sysdeps/s390/strcat.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strcat.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strcat.h | 52 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 13 +++++-
sysdeps/s390/{multiarch => }/strcat-c.c | 20 +++++----
sysdeps/s390/{multiarch => }/strcat-vx.S | 17 ++++++--
sysdeps/s390/{multiarch => }/strcat.c | 21 +++++++---
7 files changed, 108 insertions(+), 21 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strcat.h
rename sysdeps/s390/{multiarch => }/strcat-c.c (73%)
rename sysdeps/s390/{multiarch => }/strcat-vx.S (94%)
rename sysdeps/s390/{multiarch => }/strcat.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index f02afd4d02..3783f44252 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -65,5 +65,6 @@ sysdep_routines += bzero memset memset-z900 \
strcpy strcpy-vx strcpy-z900 \
stpcpy stpcpy-vx stpcpy-c \
strncpy strncpy-vx strncpy-z900 \
- stpncpy stpncpy-vx stpncpy-c
+ stpncpy stpncpy-vx stpncpy-c \
+ strcat strcat-vx strcat-c
endif
diff --git a/sysdeps/s390/ifunc-strcat.h b/sysdeps/s390/ifunc-strcat.h
new file mode 100644
index 0000000000..6fd2f7dd31
--- /dev/null
+++ b/sysdeps/s390/ifunc-strcat.h
@@ -0,0 +1,52 @@
+/* strcat variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRCAT_IFUNC 1
+#else
+# define HAVE_STRCAT_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRCAT_IFUNC_AND_VX_SUPPORT HAVE_STRCAT_IFUNC
+#else
+# define HAVE_STRCAT_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRCAT_DEFAULT STRCAT_Z13
+# define HAVE_STRCAT_C 0
+# define HAVE_STRCAT_Z13 1
+#else
+# define STRCAT_DEFAULT STRCAT_C
+# define HAVE_STRCAT_C 1
+# define HAVE_STRCAT_Z13 HAVE_STRCAT_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRCAT_C
+# define STRCAT_C __strcat_c
+#else
+# define STRCAT_C NULL
+#endif
+
+#if HAVE_STRCAT_Z13
+# define STRCAT_Z13 __strcat_vx
+#else
+# define STRCAT_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 3d97d21da1..9b66237aaf 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strcat strcat-vx strcat-c \
- strncat strncat-vx strncat-c \
+sysdep_routines += strncat strncat-vx strncat-c \
strcmp strcmp-vx \
strncmp strncmp-vx strncmp-c \
strchr strchr-vx strchr-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 021e9f247f..1b7f3df3a3 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -32,6 +32,7 @@
#include <ifunc-stpcpy.h>
#include <ifunc-strncpy.h>
#include <ifunc-stpncpy.h>
+#include <ifunc-strcat.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -242,6 +243,17 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STPNCPY_IFUNC */

+#if HAVE_STRCAT_IFUNC
+ IFUNC_IMPL (i, name, strcat,
+# if HAVE_STRCAT_Z13
+ IFUNC_IMPL_ADD (array, i, strcat,
+ dl_hwcap & HWCAP_S390_VX, STRCAT_Z13)
+# endif
+# if HAVE_STRCAT_C
+ IFUNC_IMPL_ADD (array, i, strcat, 1, STRCAT_C)
+# endif
+ )
+#endif /* HAVE_STRCAT_IFUNC */

#ifdef HAVE_S390_VX_ASM_SUPPORT

@@ -263,7 +275,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcpncpy);

- IFUNC_VX_IMPL (strcat);
IFUNC_VX_IMPL (wcscat);

IFUNC_VX_IMPL (strncat);
diff --git a/sysdeps/s390/multiarch/strcat-c.c b/sysdeps/s390/strcat-c.c
similarity index 73%
rename from sysdeps/s390/multiarch/strcat-c.c
rename to sysdeps/s390/strcat-c.c
index f871faa7b5..7accc6c7ef 100644
--- a/sysdeps/s390/multiarch/strcat-c.c
+++ b/sysdeps/s390/strcat-c.c
@@ -16,13 +16,17 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRCAT __strcat_c
-# ifdef SHARED
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
- __hidden_ver1 (__strcat_c, __GI_strcat, __strcat_c);
-# endif /* SHARED */
+#include <ifunc-strcat.h>
+
+#if HAVE_STRCAT_C
+# if HAVE_STRCAT_IFUNC
+# define STRCAT STRCAT_C
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strcat_c, __GI_strcat, __strcat_c);
+# endif
+# endif

# include <string/strcat.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strcat-vx.S b/sysdeps/s390/strcat-vx.S
similarity index 94%
rename from sysdeps/s390/multiarch/strcat-vx.S
rename to sysdeps/s390/strcat-vx.S
index 3abbbccced..218e301f10 100644
--- a/sysdeps/s390/multiarch/strcat-vx.S
+++ b/sysdeps/s390/strcat-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strcat.h>
+#if HAVE_STRCAT_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -37,7 +38,7 @@
-v17=index of zero
-v18=part of src
*/
-ENTRY(__strcat_vx)
+ENTRY(STRCAT_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -157,5 +158,13 @@ ENTRY(__strcat_vx)
vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
lgr %r2,%r0 /* Load saved dest-ptr. */
br %r14
-END(__strcat_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRCAT_Z13)
+
+# if ! HAVE_STRCAT_IFUNC
+strong_alias (STRCAT_Z13, strcat)
+# endif
+
+# if ! HAVE_STRCAT_C && defined SHARED && IS_IN (libc)
+strong_alias (STRCAT_Z13, __GI_strcat)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/strcat.c b/sysdeps/s390/strcat.c
similarity index 69%
rename from sysdeps/s390/multiarch/strcat.c
rename to sysdeps/s390/strcat.c
index 7d4126b44f..d378519c8a 100644
--- a/sysdeps/s390/multiarch/strcat.c
+++ b/sysdeps/s390/strcat.c
@@ -16,14 +16,25 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strcat.h>
+
+#if HAVE_STRCAT_IFUNC
# define strcat __redirect_strcat
# include <string.h>
# undef strcat
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strcat, __strcat, strcat)
+# if HAVE_STRCAT_C
+extern __typeof (__redirect_strcat) STRCAT_C attribute_hidden;
+# endif
+
+# if HAVE_STRCAT_Z13
+extern __typeof (__redirect_strcat) STRCAT_Z13 attribute_hidden;
+# endif

-#else
-# include <string/strcat.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_strcat, strcat,
+ (HAVE_STRCAT_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRCAT_Z13
+ : STRCAT_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for wmemchr is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.
Glibc internal calls will use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wmemchr variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wmemchr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wmemchr.
* sysdeps/s390/multiarch/wmemchr-c.c: Move to ...
* sysdeps/s390/wmemchr-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wmemchr-vx.S: Move to ...
* sysdeps/s390/wmemchr-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wmemchr.c: Move to ...
* sysdeps/s390/wmemchr.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wmemchr.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wmemchr.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wmemchr-c.c | 39 ++++++++++-------
sysdeps/s390/{multiarch => }/wmemchr-vx.S | 22 +++++++---
sysdeps/s390/{multiarch => }/wmemchr.c | 23 +++++++---
7 files changed, 126 insertions(+), 32 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wmemchr.h
rename sysdeps/s390/{multiarch => }/wmemchr-c.c (59%)
rename sysdeps/s390/{multiarch => }/wmemchr-vx.S (92%)
rename sysdeps/s390/{multiarch => }/wmemchr.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index a9058047a9..17aabf328b 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -98,5 +98,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsrchr wcsrchr-vx wcsrchr-c \
wcsspn wcsspn-vx wcsspn-c \
wcspbrk wcspbrk-vx wcspbrk-c \
- wcscspn wcscspn-vx wcscspn-c
+ wcscspn wcscspn-vx wcscspn-c \
+ wmemchr wmemchr-vx wmemchr-c
endif
diff --git a/sysdeps/s390/ifunc-wmemchr.h b/sysdeps/s390/ifunc-wmemchr.h
new file mode 100644
index 0000000000..0610cfb5d4
--- /dev/null
+++ b/sysdeps/s390/ifunc-wmemchr.h
@@ -0,0 +1,53 @@
+/* wmemchr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WMEMCHR_IFUNC 1
+#else
+# define HAVE_WMEMCHR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WMEMCHR_IFUNC_AND_VX_SUPPORT HAVE_WMEMCHR_IFUNC
+#else
+# define HAVE_WMEMCHR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WMEMCHR_DEFAULT WMEMCHR_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WMEMCHR_C 1
+# define HAVE_WMEMCHR_Z13 1
+#else
+# define WMEMCHR_DEFAULT WMEMCHR_C
+# define HAVE_WMEMCHR_C 1
+# define HAVE_WMEMCHR_Z13 HAVE_WMEMCHR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WMEMCHR_C
+# define WMEMCHR_C __wmemchr_c
+#else
+# define WMEMCHR_C NULL
+#endif
+
+#if HAVE_WMEMCHR_Z13
+# define WMEMCHR_Z13 __wmemchr_vx
+#else
+# define WMEMCHR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 5be6355423..92e28dc45d 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wmemchr wmemchr-vx wmemchr-c \
- wmemset wmemset-vx wmemset-c \
+sysdep_routines += wmemset wmemset-vx wmemset-c \
wmemcmp wmemcmp-vx wmemcmp-c
endif

diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 7d8031a069..b5f55deb7f 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -62,6 +62,7 @@
#include <ifunc-wcsspn.h>
#include <ifunc-wcspbrk.h>
#include <ifunc-wcscspn.h>
+#include <ifunc-wmemchr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -632,6 +633,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSCSPN_IFUNC */

+#if HAVE_WMEMCHR_IFUNC
+ IFUNC_IMPL (i, name, wmemchr,
+# if HAVE_WMEMCHR_Z13
+ IFUNC_IMPL_ADD (array, i, wmemchr,
+ dl_hwcap & HWCAP_S390_VX, WMEMCHR_Z13)
+# endif
+# if HAVE_WMEMCHR_C
+ IFUNC_IMPL_ADD (array, i, wmemchr, 1, WMEMCHR_C)
+# endif
+ )
+#endif /* HAVE_WMEMCHR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -640,8 +653,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wmemchr);
-
IFUNC_VX_IMPL (wmemset);

IFUNC_VX_IMPL (wmemcmp);
diff --git a/sysdeps/s390/multiarch/wmemchr-c.c b/sysdeps/s390/wmemchr-c.c
similarity index 59%
rename from sysdeps/s390/multiarch/wmemchr-c.c
rename to sysdeps/s390/wmemchr-c.c
index 089392b512..bb2526e76c 100644
--- a/sysdeps/s390/multiarch/wmemchr-c.c
+++ b/sysdeps/s390/wmemchr-c.c
@@ -16,22 +16,29 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WMEMCHR __wmemchr_c
-
-# include <wchar.h>
-extern __typeof (wmemchr) __wmemchr_c;
-# undef weak_alias
-# define weak_alias(name, alias)
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
- __hidden_ver1 (__wmemchr_c, __GI___wmemchr, __wmemchr_c);
-# undef libc_hidden_weak
-# define libc_hidden_weak(name) \
+#include <ifunc-wmemchr.h>
+
+#if HAVE_WMEMCHR_C
+# if HAVE_WMEMCHR_IFUNC || HAVE_WMEMCHR_Z13
+# define WMEMCHR WMEMCHR_C
+
+# undef weak_alias
+# define weak_alias(name, alias)
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_weak
+# define libc_hidden_weak(name)
+# undef libc_hidden_def
+# if ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wmemchr_c, __GI_wmemchr, __wmemchr_c) __attribute__((weak)); \
strong_alias (__wmemchr_c, __wmemchr_c_1); \
- __hidden_ver1 (__wmemchr_c_1, __GI_wmemchr, __wmemchr_c_1);
-# endif /* SHARED */
+ __hidden_ver1 (__wmemchr_c_1, __GI___wmemchr, __wmemchr_c_1);
+# else
+# define libc_hidden_def(name)
+# endif
+# endif
+# endif

# include <wcsmbs/wmemchr.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/wmemchr-vx.S b/sysdeps/s390/wmemchr-vx.S
similarity index 92%
rename from sysdeps/s390/multiarch/wmemchr-vx.S
rename to sysdeps/s390/wmemchr-vx.S
index db057b579a..72e9ef59af 100644
--- a/sysdeps/s390/multiarch/wmemchr-vx.S
+++ b/sysdeps/s390/wmemchr-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wmemchr.h>
+#if HAVE_WMEMCHR_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -38,7 +39,7 @@
-v17=index of found c
-v18=c replicated
*/
-ENTRY(__wmemchr_vx)
+ENTRY(WMEMCHR_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -161,6 +162,17 @@ ENTRY(__wmemchr_vx)

j .Llt64
.Lfallback:
- jg __wmemchr_c
-END(__wmemchr_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WMEMCHR_C
+END(WMEMCHR_Z13)
+
+# if ! HAVE_WMEMCHR_IFUNC
+strong_alias (WMEMCHR_Z13, __wmemchr)
+weak_alias (__wmemchr, wmemchr)
+# endif
+
+# if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT \
+ && defined SHARED && IS_IN (libc)
+strong_alias (WMEMCHR_Z13, __GI___wmemchr)
+weak_alias (WMEMCHR_Z13, __GI_wmemchr)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wmemchr.c b/sysdeps/s390/wmemchr.c
similarity index 70%
rename from sysdeps/s390/multiarch/wmemchr.c
rename to sysdeps/s390/wmemchr.c
index 6b55c1d7fa..0d2fbb22c6 100644
--- a/sysdeps/s390/multiarch/wmemchr.c
+++ b/sysdeps/s390/wmemchr.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wmemchr.h>
+
+#if HAVE_WMEMCHR_IFUNC
# define wmemchr __redirect_wmemchr
# define __wmemchr __redirect___wmemchr
# include <wchar.h>
@@ -24,9 +26,18 @@
# undef __wmemchr
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc_redirected (__redirect___wmemchr, __wmemchr)
-weak_alias (__wmemchr, wmemchr)
+# if HAVE_WMEMCHR_C
+extern __typeof (__redirect___wmemchr) WMEMCHR_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wmemchr.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WMEMCHR_Z13
+extern __typeof (__redirect___wmemchr) WMEMCHR_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect___wmemchr, __wmemchr,
+ (HAVE_WMEMCHR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WMEMCHR_Z13
+ : WMEMCHR_DEFAULT
+ )
+weak_alias (__wmemchr, wmemchr)
+#endif

[Stefan Liebler]

The ifunc handling for wcscspn is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcscspn variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcscspn variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcscspn.
* sysdeps/s390/multiarch/wcscspn-c.c: Move to ...
* sysdeps/s390/wcscspn-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcscspn-vx.S: Move to ...
* sysdeps/s390/wcscspn-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcscspn.c: Move to ...
* sysdeps/s390/wcscspn.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcscspn.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcscspn.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcscspn-c.c | 9 ++--
sysdeps/s390/{multiarch => }/wcscspn-vx.S | 15 ++++---
sysdeps/s390/{multiarch => }/wcscspn.c | 21 ++++++---
7 files changed, 100 insertions(+), 19 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcscspn.h
rename sysdeps/s390/{multiarch => }/wcscspn-c.c (86%)
rename sysdeps/s390/{multiarch => }/wcscspn-vx.S (98%)
rename sysdeps/s390/{multiarch => }/wcscspn.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 4decbf8f0a..a9058047a9 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -97,5 +97,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcschrnul wcschrnul-vx wcschrnul-c \
wcsrchr wcsrchr-vx wcsrchr-c \
wcsspn wcsspn-vx wcsspn-c \
- wcspbrk wcspbrk-vx wcspbrk-c
+ wcspbrk wcspbrk-vx wcspbrk-c \
+ wcscspn wcscspn-vx wcscspn-c
endif
diff --git a/sysdeps/s390/ifunc-wcscspn.h b/sysdeps/s390/ifunc-wcscspn.h
new file mode 100644
index 0000000000..23f3667ba3
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcscspn.h
@@ -0,0 +1,53 @@
+/* wcscspn variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSCSPN_IFUNC 1
+#else
+# define HAVE_WCSCSPN_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSCSPN_IFUNC_AND_VX_SUPPORT HAVE_WCSCSPN_IFUNC
+#else
+# define HAVE_WCSCSPN_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSCSPN_DEFAULT WCSCSPN_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSCSPN_C 1
+# define HAVE_WCSCSPN_Z13 1
+#else
+# define WCSCSPN_DEFAULT WCSCSPN_C
+# define HAVE_WCSCSPN_C 1
+# define HAVE_WCSCSPN_Z13 HAVE_WCSCSPN_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSCSPN_C
+# define WCSCSPN_C __wcscspn_c
+#else
+# define WCSCSPN_C NULL
+#endif
+
+#if HAVE_WCSCSPN_Z13
+# define WCSCSPN_Z13 __wcscspn_vx
+#else
+# define WCSCSPN_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index e1e2d9dc74..5be6355423 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcscspn wcscspn-vx wcscspn-c \
- wmemchr wmemchr-vx wmemchr-c \
+sysdep_routines += wmemchr wmemchr-vx wmemchr-c \
wmemset wmemset-vx wmemset-c \
wmemcmp wmemcmp-vx wmemcmp-c
endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 89d6e8ad7e..7d8031a069 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -61,6 +61,7 @@
#include <ifunc-wcsrchr.h>
#include <ifunc-wcsspn.h>
#include <ifunc-wcspbrk.h>
+#include <ifunc-wcscspn.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -619,6 +620,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSPBRK_IFUNC */

+#if HAVE_WCSCSPN_IFUNC
+ IFUNC_IMPL (i, name, wcscspn,
+# if HAVE_WCSCSPN_Z13
+ IFUNC_IMPL_ADD (array, i, wcscspn,
+ dl_hwcap & HWCAP_S390_VX, WCSCSPN_Z13)
+# endif
+# if HAVE_WCSCSPN_C
+ IFUNC_IMPL_ADD (array, i, wcscspn, 1, WCSCSPN_C)
+# endif
+ )
+#endif /* HAVE_WCSCSPN_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -627,8 +640,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcscspn);
-
IFUNC_VX_IMPL (wmemchr);

IFUNC_VX_IMPL (wmemset);
diff --git a/sysdeps/s390/multiarch/wcscspn-c.c b/sysdeps/s390/wcscspn-c.c
similarity index 86%
rename from sysdeps/s390/multiarch/wcscspn-c.c
rename to sysdeps/s390/wcscspn-c.c
index 161e52e686..d47cb6b75b 100644
--- a/sysdeps/s390/multiarch/wcscspn-c.c
+++ b/sysdeps/s390/wcscspn-c.c
@@ -16,11 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSCSPN __wcscspn_c
+#include <ifunc-wcscspn.h>

-# include <wchar.h>
-extern __typeof (wcscspn) __wcscspn_c;
+#if HAVE_WCSCSPN_C
+# if HAVE_WCSCSPN_IFUNC || HAVE_WCSCSPN_Z13
+# define WCSCSPN WCSCSPN_C
+# endif

# include <wcsmbs/wcscspn.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcscspn-vx.S b/sysdeps/s390/wcscspn-vx.S
similarity index 98%
rename from sysdeps/s390/multiarch/wcscspn-vx.S
rename to sysdeps/s390/wcscspn-vx.S
index 06bc4e25d0..882cb93fb8 100644
--- a/sysdeps/s390/multiarch/wcscspn-vx.S
+++ b/sysdeps/s390/wcscspn-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcscspn.h>
+#if HAVE_WCSCSPN_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -57,7 +58,7 @@
otherwise =0;
r9: loaded byte count of vlbb reject-string
*/
-ENTRY(__wcscspn_vx)
+ENTRY(WCSCSPN_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -288,6 +289,10 @@ ENTRY(__wcscspn_vx)
vlgvg %r9,%v31,1
br %r14
.Lfallback:
- jg __wcscspn_c
-END(__wcscspn_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSCSPN_C
+END(WCSCSPN_Z13)
+
+# if ! HAVE_WCSCSPN_IFUNC
+strong_alias (WCSCSPN_Z13, wcscspn)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcscspn.c b/sysdeps/s390/wcscspn.c
similarity index 69%
rename from sysdeps/s390/multiarch/wcscspn.c
rename to sysdeps/s390/wcscspn.c
index 707327522a..0ce31b8aab 100644
--- a/sysdeps/s390/multiarch/wcscspn.c
+++ b/sysdeps/s390/wcscspn.c
@@ -16,12 +16,23 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcscspn.h>
+
+#if HAVE_WCSCSPN_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2 (__wcscspn, wcscspn)
+# if HAVE_WCSCSPN_C
+extern __typeof (wcscspn) WCSCSPN_C attribute_hidden;
+# endif
+
+# if HAVE_WCSCSPN_Z13
+extern __typeof (wcscspn) WCSCSPN_Z13 attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcscspn.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (wcscspn, wcscspn,
+ (HAVE_WCSCSPN_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSCSPN_Z13
+ : WCSCSPN_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for wcsrchr is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcsrchr variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcsrchr variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcsrchr.
* sysdeps/s390/multiarch/wcsrchr-c.c: Move to ...
* sysdeps/s390/wcsrchr-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsrchr-vx.S: Move to ...
* sysdeps/s390/wcsrchr-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsrchr.c: Move to ...
* sysdeps/s390/wcsrchr.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcsrchr.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcsrchr.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcsrchr-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcsrchr-vx.S | 15 ++++---
sysdeps/s390/{multiarch => }/wcsrchr.c | 21 ++++++---
7 files changed, 101 insertions(+), 19 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcsrchr.h
rename sysdeps/s390/{multiarch => }/wcsrchr-c.c (85%)
rename sysdeps/s390/{multiarch => }/wcsrchr-vx.S (97%)
rename sysdeps/s390/{multiarch => }/wcsrchr.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index a8cd264521..a2c641e81f 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -94,5 +94,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcscmp wcscmp-vx wcscmp-c \
wcsncmp wcsncmp-vx wcsncmp-c \
wcschr wcschr-vx wcschr-c \
- wcschrnul wcschrnul-vx wcschrnul-c
+ wcschrnul wcschrnul-vx wcschrnul-c \
+ wcsrchr wcsrchr-vx wcsrchr-c
endif
diff --git a/sysdeps/s390/ifunc-wcsrchr.h b/sysdeps/s390/ifunc-wcsrchr.h
new file mode 100644
index 0000000000..4eec0c1707
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcsrchr.h
@@ -0,0 +1,53 @@
+/* wcsrchr variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSRCHR_IFUNC 1
+#else
+# define HAVE_WCSRCHR_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSRCHR_IFUNC_AND_VX_SUPPORT HAVE_WCSRCHR_IFUNC
+#else
+# define HAVE_WCSRCHR_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSRCHR_DEFAULT WCSRCHR_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSRCHR_C 1
+# define HAVE_WCSRCHR_Z13 1
+#else
+# define WCSRCHR_DEFAULT WCSRCHR_C
+# define HAVE_WCSRCHR_C 1
+# define HAVE_WCSRCHR_Z13 HAVE_WCSRCHR_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSRCHR_C
+# define WCSRCHR_C __wcsrchr_c
+#else
+# define WCSRCHR_C NULL
+#endif
+
+#if HAVE_WCSRCHR_Z13
+# define WCSRCHR_Z13 __wcsrchr_vx
+#else
+# define WCSRCHR_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 0eb01d5d84..3d72cbb5f3 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcsrchr wcsrchr-vx wcsrchr-c \
- wcsspn wcsspn-vx wcsspn-c \
+sysdep_routines += wcsspn wcsspn-vx wcsspn-c \
wcspbrk wcspbrk-vx wcspbrk-c \
wcscspn wcscspn-vx wcscspn-c \
wmemchr wmemchr-vx wmemchr-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 553a76c39e..ffb4d5b687 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -58,6 +58,7 @@
#include <ifunc-wcsncmp.h>
#include <ifunc-wcschr.h>
#include <ifunc-wcschrnul.h>
+#include <ifunc-wcsrchr.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -580,6 +581,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSCHRNUL_IFUNC */

+#if HAVE_WCSRCHR_IFUNC
+ IFUNC_IMPL (i, name, wcsrchr,
+# if HAVE_WCSRCHR_Z13
+ IFUNC_IMPL_ADD (array, i, wcsrchr,
+ dl_hwcap & HWCAP_S390_VX, WCSRCHR_Z13)
+# endif
+# if HAVE_WCSRCHR_C
+ IFUNC_IMPL_ADD (array, i, wcsrchr, 1, WCSRCHR_C)
+# endif
+ )
+#endif /* HAVE_WCSRCHR_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -588,8 +601,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcsrchr);
-
IFUNC_VX_IMPL (wcsspn);

IFUNC_VX_IMPL (wcspbrk);
diff --git a/sysdeps/s390/multiarch/wcsrchr-c.c b/sysdeps/s390/wcsrchr-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/wcsrchr-c.c
rename to sysdeps/s390/wcsrchr-c.c
index 615250358b..65164a897f 100644
--- a/sysdeps/s390/multiarch/wcsrchr-c.c
+++ b/sysdeps/s390/wcsrchr-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSRCHR __wcsrchr_c
+#include <ifunc-wcsrchr.h>
+
+#if HAVE_WCSRCHR_C
+# if HAVE_WCSRCHR_IFUNC || HAVE_WCSRCHR_Z13
+# define WCSRCHR WCSRCHR_C
+# endif

-# include <wchar.h>
-extern __typeof (wcsrchr) __wcsrchr_c;
# include <wcsmbs/wcsrchr.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcsrchr-vx.S b/sysdeps/s390/wcsrchr-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/wcsrchr-vx.S
rename to sysdeps/s390/wcsrchr-vx.S
index e40a554e5d..c7b6585bea 100644
--- a/sysdeps/s390/multiarch/wcsrchr-vx.S
+++ b/sysdeps/s390/wcsrchr-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsrchr.h>
+#if HAVE_WCSRCHR_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -39,7 +40,7 @@
-v19=part of s with last occurence of c.
-v20=permute pattern
*/
-ENTRY(__wcsrchr_vx)
+ENTRY(WCSRCHR_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -185,6 +186,10 @@ ENTRY(__wcsrchr_vx)
.byte 0x0C,0x0D,0x0E,0x0F,0x08,0x09,0x0A,0x0B
.byte 0x04,0x05,0x06,0x07,0x00,0x01,0x02,0x03
.Lfallback:
- jg __wcsrchr_c
-END(__wcsrchr_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSRCHR_C
+END(WCSRCHR_Z13)
+
+# if ! HAVE_WCSRCHR_IFUNC
+strong_alias (WCSRCHR_Z13, wcsrchr)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcsrchr.c b/sysdeps/s390/wcsrchr.c
similarity index 69%
rename from sysdeps/s390/multiarch/wcsrchr.c
rename to sysdeps/s390/wcsrchr.c
index aa0b8a8f82..eb70f27038 100644
--- a/sysdeps/s390/multiarch/wcsrchr.c
+++ b/sysdeps/s390/wcsrchr.c
@@ -16,12 +16,23 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsrchr.h>
+
+#if HAVE_WCSRCHR_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2 (__wcsrchr, wcsrchr)
+# if HAVE_WCSRCHR_C
+extern __typeof (wcsrchr) WCSRCHR_C attribute_hidden;
+# endif
+
+# if HAVE_WCSRCHR_Z13
+extern __typeof (wcsrchr) WCSRCHR_Z13 attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcsrchr.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (wcsrchr, wcsrchr,
+ (HAVE_WCSRCHR_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSRCHR_Z13
+ : WCSRCHR_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for wcsspn is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.
Glibc internal calls will use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcsspn variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcsspn variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcsspn.
* sysdeps/s390/multiarch/wcsspn-c.c: Move to ...
* sysdeps/s390/wcsspn-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsspn-vx.S: Move to ...
* sysdeps/s390/wcsspn-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsspn.c: Move to ...
* sysdeps/s390/wcsspn.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcsspn.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcsspn.h | 53 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcsspn-c.c | 24 +++++++----
sysdeps/s390/{multiarch => }/wcsspn-vx.S | 20 ++++++---
sysdeps/s390/{multiarch => }/wcsspn.c | 20 ++++++---
7 files changed, 114 insertions(+), 24 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcsspn.h
rename sysdeps/s390/{multiarch => }/wcsspn-c.c (72%)
rename sysdeps/s390/{multiarch => }/wcsspn-vx.S (97%)
rename sysdeps/s390/{multiarch => }/wcsspn.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index a2c641e81f..2e48900c68 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -95,5 +95,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsncmp wcsncmp-vx wcsncmp-c \
wcschr wcschr-vx wcschr-c \
wcschrnul wcschrnul-vx wcschrnul-c \
- wcsrchr wcsrchr-vx wcsrchr-c
+ wcsrchr wcsrchr-vx wcsrchr-c \
+ wcsspn wcsspn-vx wcsspn-c
endif
diff --git a/sysdeps/s390/ifunc-wcsspn.h b/sysdeps/s390/ifunc-wcsspn.h
new file mode 100644
index 0000000000..1189c6b93f
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcsspn.h
@@ -0,0 +1,53 @@
+/* wcsspn variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSSPN_IFUNC 1
+#else
+# define HAVE_WCSSPN_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSSPN_IFUNC_AND_VX_SUPPORT HAVE_WCSSPN_IFUNC
+#else
+# define HAVE_WCSSPN_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSSPN_DEFAULT WCSSPN_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSSPN_C 1
+# define HAVE_WCSSPN_Z13 1
+#else
+# define WCSSPN_DEFAULT WCSSPN_C
+# define HAVE_WCSSPN_C 1
+# define HAVE_WCSSPN_Z13 HAVE_WCSSPN_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSSPN_C
+# define WCSSPN_C __wcsspn_c
+#else
+# define WCSSPN_C NULL
+#endif
+
+#if HAVE_WCSSPN_Z13
+# define WCSSPN_Z13 __wcsspn_vx
+#else
+# define WCSSPN_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 3d72cbb5f3..091f515084 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcsspn wcsspn-vx wcsspn-c \
- wcspbrk wcspbrk-vx wcspbrk-c \
+sysdep_routines += wcspbrk wcspbrk-vx wcspbrk-c \
wcscspn wcscspn-vx wcscspn-c \
wmemchr wmemchr-vx wmemchr-c \
wmemset wmemset-vx wmemset-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index ffb4d5b687..6f4de2845b 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -59,6 +59,7 @@
#include <ifunc-wcschr.h>
#include <ifunc-wcschrnul.h>
#include <ifunc-wcsrchr.h>
+#include <ifunc-wcsspn.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -593,6 +594,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSRCHR_IFUNC */

+#if HAVE_WCSSPN_IFUNC
+ IFUNC_IMPL (i, name, wcsspn,
+# if HAVE_WCSSPN_Z13
+ IFUNC_IMPL_ADD (array, i, wcsspn,
+ dl_hwcap & HWCAP_S390_VX, WCSSPN_Z13)
+# endif
+# if HAVE_WCSSPN_C
+ IFUNC_IMPL_ADD (array, i, wcsspn, 1, WCSSPN_C)
+# endif
+ )
+#endif /* HAVE_WCSSPN_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -601,8 +614,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcsspn);
-
IFUNC_VX_IMPL (wcspbrk);

IFUNC_VX_IMPL (wcscspn);
diff --git a/sysdeps/s390/multiarch/wcsspn-c.c b/sysdeps/s390/wcsspn-c.c
similarity index 72%
rename from sysdeps/s390/multiarch/wcsspn-c.c
rename to sysdeps/s390/wcsspn-c.c
index 2c0bd0f4e6..db3bdb9b9f 100644
--- a/sysdeps/s390/multiarch/wcsspn-c.c
+++ b/sysdeps/s390/wcsspn-c.c
@@ -16,16 +16,22 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSSPN __wcsspn_c
+#include <ifunc-wcsspn.h>

-# include <wchar.h>
-extern __typeof (wcsspn) __wcsspn_c;
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
+#if HAVE_WCSSPN_C
+# if HAVE_WCSSPN_IFUNC || HAVE_WCSSPN_Z13
+# define WCSSPN WCSSPN_C
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# if ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define libc_hidden_def(name) \
__hidden_ver1 (__wcsspn_c, __GI_wcsspn, __wcsspn_c);
-# endif /* SHARED */
+# else
+# define libc_hidden_def(name)
+# endif
+# endif
+# endif

# include <wcsmbs/wcsspn.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/wcsspn-vx.S b/sysdeps/s390/wcsspn-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/wcsspn-vx.S
rename to sysdeps/s390/wcsspn-vx.S
index 548f2ad164..61f7d6d090 100644
--- a/sysdeps/s390/multiarch/wcsspn-vx.S
+++ b/sysdeps/s390/wcsspn-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsspn.h>
+#if HAVE_WCSSPN_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -57,7 +58,7 @@
otherwise =0;
r9: loaded byte count of vlbb accept-string
*/
-ENTRY(__wcsspn_vx)
+ENTRY(WCSSPN_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -265,6 +266,15 @@ ENTRY(__wcsspn_vx)
j .Lslow_next_acc_notonbb /* ... and search for zero in
fully loaded vreg again. */
.Lfallback:
- jg __wcsspn_c
-END(__wcsspn_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSSPN_C
+END(WCSSPN_Z13)
+
+# if ! HAVE_WCSSPN_IFUNC
+strong_alias (WCSSPN_Z13, wcsspn)
+# endif
+
+# if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT \
+ && defined SHARED && IS_IN (libc)
+strong_alias (WCSSPN_Z13, __GI_wcsspn)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcsspn.c b/sysdeps/s390/wcsspn.c
similarity index 69%
rename from sysdeps/s390/multiarch/wcsspn.c
rename to sysdeps/s390/wcsspn.c
index 7743144a8c..e916b1e441 100644
--- a/sysdeps/s390/multiarch/wcsspn.c
+++ b/sysdeps/s390/wcsspn.c
@@ -16,14 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsspn.h>
+
+#if HAVE_WCSSPN_IFUNC
# define wcsspn __redirect_wcsspn
# include <wchar.h>
# undef wcsspn
# include <ifunc-resolve.h>
+# if HAVE_WCSSPN_C
+extern __typeof (__redirect_wcsspn) WCSSPN_C attribute_hidden;
+# endif

-s390_vx_libc_ifunc2_redirected (__redirect_wcsspn, __wcsspn, wcsspn)
+# if HAVE_WCSSPN_Z13
+extern __typeof (__redirect_wcsspn) WCSSPN_Z13 attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcsspn.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_wcsspn, wcsspn,
+ (HAVE_WCSSPN_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSSPN_Z13
+ : WCSSPN_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for strcspn is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strcspn variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strcspn variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strcspn.
* sysdeps/s390/multiarch/strcspn-c.c: Move to ...
* sysdeps/s390/strcspn-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strcspn-vx.S: Move to ...
* sysdeps/s390/strcspn-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strcspn.c: Move to ...
* sysdeps/s390/strcspn.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strcspn.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strcspn.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 +++++-
sysdeps/s390/{multiarch => }/strcspn-c.c | 20 +++++----
sysdeps/s390/{multiarch => }/strcspn-vx.S | 19 +++++++--
sysdeps/s390/{multiarch => }/strcspn.c | 21 ++++++---
7 files changed, 111 insertions(+), 21 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strcspn.h
rename sysdeps/s390/{multiarch => }/strcspn-c.c (73%)
rename sysdeps/s390/{multiarch => }/strcspn-vx.S (97%)
rename sysdeps/s390/{multiarch => }/strcspn.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index df1f858e28..c163969652 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -74,5 +74,6 @@ sysdep_routines += bzero memset memset-z900 \
strchrnul strchrnul-vx strchrnul-c \
strrchr strrchr-vx strrchr-c \
strspn strspn-vx strspn-c \
- strpbrk strpbrk-vx strpbrk-c
+ strpbrk strpbrk-vx strpbrk-c \
+ strcspn strcspn-vx strcspn-c
endif
diff --git a/sysdeps/s390/ifunc-strcspn.h b/sysdeps/s390/ifunc-strcspn.h
new file mode 100644
index 0000000000..9b7032509a
--- /dev/null
+++ b/sysdeps/s390/ifunc-strcspn.h
@@ -0,0 +1,52 @@
+/* strcspn variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRCSPN_IFUNC 1
+#else
+# define HAVE_STRCSPN_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRCSPN_IFUNC_AND_VX_SUPPORT HAVE_STRCSPN_IFUNC
+#else
+# define HAVE_STRCSPN_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRCSPN_DEFAULT STRCSPN_Z13
+# define HAVE_STRCSPN_C 0
+# define HAVE_STRCSPN_Z13 1
+#else
+# define STRCSPN_DEFAULT STRCSPN_C
+# define HAVE_STRCSPN_C 1
+# define HAVE_STRCSPN_Z13 HAVE_STRCSPN_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRCSPN_C
+# define STRCSPN_C __strcspn_c
+#else
+# define STRCSPN_C NULL
+#endif
+
+#if HAVE_STRCSPN_Z13
+# define STRCSPN_Z13 __strcspn_vx
+#else
+# define STRCSPN_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 1a3fed9fc8..1578f21af4 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strcspn strcspn-vx strcspn-c \
- memchr memchr-vx \
+sysdep_routines += memchr memchr-vx \
rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
memrchr memrchr-vx memrchr-c
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 8e23416730..2d48c99c8d 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -41,6 +41,7 @@
#include <ifunc-strrchr.h>
#include <ifunc-strspn.h>
#include <ifunc-strpbrk.h>
+#include <ifunc-strcspn.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -359,6 +360,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRPBRK_IFUNC */

+#if HAVE_STRCSPN_IFUNC
+ IFUNC_IMPL (i, name, strcspn,
+# if HAVE_STRCSPN_Z13
+ IFUNC_IMPL_ADD (array, i, strcspn,
+ dl_hwcap & HWCAP_S390_VX, STRCSPN_Z13)
+# endif
+# if HAVE_STRCSPN_C
+ IFUNC_IMPL_ADD (array, i, strcspn, 1, STRCSPN_C)
+# endif
+ )
+#endif /* HAVE_STRCSPN_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -397,7 +410,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcspbrk);

- IFUNC_VX_IMPL (strcspn);
IFUNC_VX_IMPL (wcscspn);

IFUNC_VX_IMPL (memchr);
diff --git a/sysdeps/s390/multiarch/strcspn-c.c b/sysdeps/s390/strcspn-c.c
similarity index 73%
rename from sysdeps/s390/multiarch/strcspn-c.c
rename to sysdeps/s390/strcspn-c.c
index 7b454f5b56..9f51f92bdb 100644
--- a/sysdeps/s390/multiarch/strcspn-c.c
+++ b/sysdeps/s390/strcspn-c.c
@@ -16,13 +16,17 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRCSPN __strcspn_c
-# ifdef SHARED
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
- __hidden_ver1 (__strcspn_c, __GI_strcspn, __strcspn_c);
-# endif /* SHARED */
+#include <ifunc-strcspn.h>
+
+#if HAVE_STRCSPN_C
+# if HAVE_STRCSPN_IFUNC
+# define STRCSPN STRCSPN_C
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
+ __hidden_ver1 (__strcspn_c, __GI_strcspn, __strcspn_c);
+# endif
+# endif

# include <string/strcspn.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strcspn-vx.S b/sysdeps/s390/strcspn-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/strcspn-vx.S
rename to sysdeps/s390/strcspn-vx.S
index ea1668742b..ff5b1be549 100644
--- a/sysdeps/s390/multiarch/strcspn-vx.S
+++ b/sysdeps/s390/strcspn-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strcspn.h>
+
+#if HAVE_STRCSPN_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -58,7 +60,7 @@
otherwise =0;
r9: loaded byte count of vlbb reject-string
*/
-ENTRY(__strcspn_vx)
+ENTRY(STRCSPN_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -277,5 +279,14 @@ ENTRY(__strcspn_vx)
vlgvg %r8,%v31,0
vlgvg %r9,%v31,1
br %r14
-END(__strcspn_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRCSPN_Z13)
+
+# if ! HAVE_STRCSPN_IFUNC
+strong_alias (STRCSPN_Z13, strcspn)
+# endif
+
+# if ! HAVE_STRCSPN_C && defined SHARED && IS_IN (libc)
+strong_alias (STRCSPN_Z13, __GI_strcspn)
+# endif
+
+#endif /* HAVE_STRCSPN_Z13 */
diff --git a/sysdeps/s390/multiarch/strcspn.c b/sysdeps/s390/strcspn.c
similarity index 70%
rename from sysdeps/s390/multiarch/strcspn.c
rename to sysdeps/s390/strcspn.c
index 418ffcdded..a3f35d39c5 100644
--- a/sysdeps/s390/multiarch/strcspn.c
+++ b/sysdeps/s390/strcspn.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strcspn.h>
+
+#if HAVE_STRCSPN_IFUNC
# define strcspn __redirect_strcspn
/* Omit the strcspn inline definitions because it would redefine strcspn. */
# define __NO_STRING_INLINES
@@ -24,8 +26,17 @@
# undef strcspn
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strcspn, __strcspn, strcspn)
+# if HAVE_STRCSPN_C
+extern __typeof (__redirect_strcspn) STRCSPN_C attribute_hidden;
+# endif
+
+# if HAVE_STRCSPN_Z13
+extern __typeof (__redirect_strcspn) STRCSPN_Z13 attribute_hidden;
+# endif

-#else
-# include <string/strcspn.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_strcspn, strcspn,
+ (HAVE_STRCSPN_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRCSPN_Z13
+ : STRCSPN_DEFAULT
+ )
+#endif /* HAVE_STRCSPN_IFUNC */

[Stefan Liebler]

The new vector variant of memmem is using the common code
implementation, but instead of calling the default
mem* functions, the vector variants are called.

ChangeLog:

* sysdeps/s390/Makefile (sysdep_routines): Add memmem variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc variants for memmem.
* sysdeps/s390/ifunc-memmem.h: New file.
* sysdeps/s390/memmem.c: Likewise.
* sysdeps/s390/memmem-c.c: Likewise.
* sysdeps/s390/memmem-vx.c: Likewise.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-memmem.h | 52 ++++++++++++++++++++
sysdeps/s390/memmem-c.c | 47 ++++++++++++++++++
sysdeps/s390/memmem-vx.c | 61 ++++++++++++++++++++++++
sysdeps/s390/memmem.c | 43 +++++++++++++++++
sysdeps/s390/multiarch/ifunc-impl-list.c | 13 +++++
6 files changed, 218 insertions(+), 1 deletion(-)
create mode 100644 sysdeps/s390/ifunc-memmem.h
create mode 100644 sysdeps/s390/memmem-c.c
create mode 100644 sysdeps/s390/memmem-vx.c
create mode 100644 sysdeps/s390/memmem.c

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index c886fe31f3..0a964e0d4d 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -58,5 +58,6 @@ sysdep_routines += bzero memset memset-z900 \
memcmp memcmp-z900 \
mempcpy memcpy memcpy-z900 \
memmove memmove-c \
- strstr strstr-vx strstr-c
+ strstr strstr-vx strstr-c \
+ memmem memmem-vx memmem-c
endif
diff --git a/sysdeps/s390/ifunc-memmem.h b/sysdeps/s390/ifunc-memmem.h
new file mode 100644
index 0000000000..0f860d8d40
--- /dev/null
+++ b/sysdeps/s390/ifunc-memmem.h
@@ -0,0 +1,52 @@
+/* memmem variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_MEMMEM_IFUNC 1
+#else
+# define HAVE_MEMMEM_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_MEMMEM_IFUNC_AND_VX_SUPPORT HAVE_MEMMEM_IFUNC
+#else
+# define HAVE_MEMMEM_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define MEMMEM_DEFAULT MEMMEM_Z13
+# define HAVE_MEMMEM_C 0
+# define HAVE_MEMMEM_Z13 1
+#else
+# define MEMMEM_DEFAULT MEMMEM_C
+# define HAVE_MEMMEM_C 1
+# define HAVE_MEMMEM_Z13 HAVE_MEMMEM_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_MEMMEM_C
+# define MEMMEM_C __memmem_c
+#else
+# define MEMMEM_C NULL
+#endif
+
+#if HAVE_MEMMEM_Z13
+# define MEMMEM_Z13 __memmem_vx
+#else
+# define MEMMEM_Z13 NULL
+#endif
diff --git a/sysdeps/s390/memmem-c.c b/sysdeps/s390/memmem-c.c
new file mode 100644
index 0000000000..1d8ffefcb8
--- /dev/null
+++ b/sysdeps/s390/memmem-c.c
@@ -0,0 +1,47 @@
+/* Default memmem implementation for S/390.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <ifunc-memmem.h>
+
+#if HAVE_MEMMEM_C
+# if HAVE_MEMMEM_IFUNC
+# include <string.h>
+
+# ifndef _LIBC
+# define memmem MEMMEM_C
+# else
+# define __memmem MEMMEM_C
+# endif
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
+ strong_alias (__memmem_c, __memmem_c_1); \
+ __hidden_ver1 (__memmem_c, __GI___memmem, __memmem_c);
+
+# undef libc_hidden_weak
+# define libc_hidden_weak(name) \
+ __hidden_ver1 (__memmem_c_1, __GI_memmem, __memmem_c_1) __attribute__((weak));
+# endif
+
+# undef weak_alias
+# define weak_alias(a, b)
+# endif
+
+# include <string/memmem.c>
+#endif
diff --git a/sysdeps/s390/memmem-vx.c b/sysdeps/s390/memmem-vx.c
new file mode 100644
index 0000000000..af6e200e4e
--- /dev/null
+++ b/sysdeps/s390/memmem-vx.c
@@ -0,0 +1,61 @@
+/* Default memmem implementation with vector string functions for S/390.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <ifunc-memmem.h>
+
+#if HAVE_MEMMEM_Z13
+# include <string.h>
+# if HAVE_MEMMEM_IFUNC
+
+# ifndef _LIBC
+# define memmem MEMMEM_Z13
+# else
+# define __memmem MEMMEM_Z13
+# endif
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# undef libc_hidden_weak
+
+# if HAVE_MEMMEM_C
+# define libc_hidden_def(name)
+# define libc_hidden_weak(name)
+# else
+# define libc_hidden_def(name) \
+ strong_alias (__memmem_vx, __memmem_vx_1); \
+ __hidden_ver1 (__memmem_vx, __GI___memmem, __memmem_vx);
+
+# define libc_hidden_weak(name) \
+ __hidden_ver1 (__memmem_vx_1, __GI_memmem, __memmem_vx_1) __attribute__((weak));
+# endif
+# endif
+
+# undef weak_alias
+# define weak_alias(a, b)
+# endif
+
+# ifdef USE_MULTIARCH
+extern __typeof (memchr) __memchr_vx attribute_hidden;
+# define memchr __memchr_vx
+
+extern __typeof (memcmp) __memcmp_z196 attribute_hidden;
+# define memcmp __memcmp_z196
+# endif
+
+# include <string/memmem.c>
+#endif
diff --git a/sysdeps/s390/memmem.c b/sysdeps/s390/memmem.c
new file mode 100644
index 0000000000..8c50b3f403
--- /dev/null
+++ b/sysdeps/s390/memmem.c
@@ -0,0 +1,43 @@
+/* Multiple versions of memmem.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <ifunc-memmem.h>
+
+#if HAVE_MEMMEM_IFUNC
+# define memmem __redirect_memmem
+# define __memmem __redirect___memmem
+# include <string.h>
+# include <ifunc-resolve.h>
+# undef memmem
+# undef __memmem
+
+# if HAVE_MEMMEM_C
+extern __typeof (__redirect_memmem) MEMMEM_C attribute_hidden;
+# endif
+
+# if HAVE_MEMMEM_Z13
+extern __typeof (__redirect_memmem) MEMMEM_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect_memmem, __memmem,
+ (HAVE_MEMMEM_Z13 && (hwcap & HWCAP_S390_VX))
+ ? MEMMEM_Z13
+ : MEMMEM_DEFAULT
+ )
+weak_alias (__memmem, memmem)
+#endif
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 14727f8fef..da8696d917 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -25,6 +25,7 @@
#include <ifunc-memcmp.h>
#include <ifunc-memcpy.h>
#include <ifunc-strstr.h>
+#include <ifunc-memmem.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -151,6 +152,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRSTR_IFUNC */

+#if HAVE_MEMMEM_IFUNC
+ IFUNC_IMPL (i, name, memmem,
+# if HAVE_MEMMEM_Z13
+ IFUNC_IMPL_ADD (array, i, memmem,
+ dl_hwcap & HWCAP_S390_VX, MEMMEM_Z13)
+# endif
+# if HAVE_MEMMEM_C
+ IFUNC_IMPL_ADD (array, i, memmem, 1, MEMMEM_C)
+# endif
+ )
+#endif /* HAVE_MEMMEM_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \

[Stefan Liebler]

The ifunc handling for various __gconv_transform_* functions
which are using IFUNC on s390x are adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Instead those functions are just an alias to the vector variants.

Furthermore the ifunc-macro s390_libc_ifunc_expr is now used instead of
s390_vx_libc_ifunc.

ChangeLog:

* sysdeps/s390/multiarch/gconv_simple.c (ICONV_VX_IFUNC):
Define macro dependent on HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT.
---
sysdeps/s390/multiarch/gconv_simple.c | 23 ++++++++++++-----------
1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/sysdeps/s390/multiarch/gconv_simple.c b/sysdeps/s390/multiarch/gconv_simple.c
index aaa1ebf74a..078d992c13 100644
--- a/sysdeps/s390/multiarch/gconv_simple.c
+++ b/sysdeps/s390/multiarch/gconv_simple.c
@@ -27,17 +27,18 @@

# define ICONV_C_NAME(NAME) __##NAME##_c
# define ICONV_VX_NAME(NAME) __##NAME##_vx
-# define ICONV_VX_IFUNC(FUNC) \
- extern __typeof (ICONV_C_NAME (FUNC)) __##FUNC; \
- s390_vx_libc_ifunc (__##FUNC) \
- int FUNC (struct __gconv_step *step, struct __gconv_step_data *data, \
- const unsigned char **inptrp, const unsigned char *inend, \
- unsigned char **outbufstart, size_t *irreversible, \
- int do_flush, int consume_incomplete) \
- { \
- return __##FUNC (step, data, inptrp, inend,outbufstart, \
- irreversible, do_flush, consume_incomplete); \
- }
+# ifdef HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+/* We support z13 instructions by default -> Just use the vector variant. */
+# define ICONV_VX_IFUNC(FUNC) strong_alias (ICONV_VX_NAME (FUNC), FUNC)
+# else
+/* We have to use ifunc to determine if z13 instructions are supported. */
+# define ICONV_VX_IFUNC(FUNC) \
+ s390_libc_ifunc_expr (ICONV_C_NAME (FUNC), FUNC, \
+ (hwcap & HWCAP_S390_VX) \
+ ? ICONV_VX_NAME (FUNC) \
+ : ICONV_C_NAME (FUNC) \
+ )
+# endif
# define ICONV_VX_SINGLE(NAME) \
static __typeof (NAME##_single) __##NAME##_vx_single __attribute__((alias(#NAME "_single")));

[Stefan Liebler]

The ifunc handling for wcscmp is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcscmp variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcscmp variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcscmp.
* sysdeps/s390/multiarch/wcscmp-c.c: Move to ...
* sysdeps/s390/wcscmp-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcscmp-vx.S: Move to ...
* sysdeps/s390/wcscmp-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcscmp.c: Move to ...
* sysdeps/s390/wcscmp.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcscmp.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcscmp.h | 52 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcscmp-c.c | 24 ++++++-----
sysdeps/s390/{multiarch => }/wcscmp-vx.S | 18 ++++++--
sysdeps/s390/{multiarch => }/wcscmp.c | 23 ++++++++---
7 files changed, 112 insertions(+), 26 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcscmp.h
rename sysdeps/s390/{multiarch => }/wcscmp-c.c (72%)
rename sysdeps/s390/{multiarch => }/wcscmp-vx.S (92%)
rename sysdeps/s390/{multiarch => }/wcscmp.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index cb8dd2134f..00398ba6c2 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -90,5 +90,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsncpy wcsncpy-vx wcsncpy-c \
wcpncpy wcpncpy-vx wcpncpy-c \
wcscat wcscat-vx wcscat-c \
- wcsncat wcsncat-vx wcsncat-c
+ wcsncat wcsncat-vx wcsncat-c \
+ wcscmp wcscmp-vx wcscmp-c
endif
diff --git a/sysdeps/s390/ifunc-wcscmp.h b/sysdeps/s390/ifunc-wcscmp.h
new file mode 100644
index 0000000000..99fe021302
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcscmp.h
@@ -0,0 +1,52 @@
+/* wcscmp variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSCMP_IFUNC 1
+#else
+# define HAVE_WCSCMP_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSCMP_IFUNC_AND_VX_SUPPORT HAVE_WCSCMP_IFUNC
+#else
+# define HAVE_WCSCMP_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSCMP_DEFAULT WCSCMP_Z13
+# define HAVE_WCSCMP_C 0
+# define HAVE_WCSCMP_Z13 1
+#else
+# define WCSCMP_DEFAULT WCSCMP_C
+# define HAVE_WCSCMP_C 1
+# define HAVE_WCSCMP_Z13 HAVE_WCSCMP_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSCMP_C
+# define WCSCMP_C __wcscmp_c
+#else
+# define WCSCMP_C NULL
+#endif
+
+#if HAVE_WCSCMP_Z13
+# define WCSCMP_Z13 __wcscmp_vx
+#else
+# define WCSCMP_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 6cb75950c2..70162d0eaa 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcscmp wcscmp-vx wcscmp-c \
- wcsncmp wcsncmp-vx wcsncmp-c \
+sysdep_routines += wcsncmp wcsncmp-vx wcsncmp-c \
wcschr wcschr-vx wcschr-c \
wcschrnul wcschrnul-vx wcschrnul-c \
wcsrchr wcsrchr-vx wcsrchr-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 7b7b1e7497..f461063c80 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -54,6 +54,7 @@
#include <ifunc-wcpncpy.h>
#include <ifunc-wcscat.h>
#include <ifunc-wcsncat.h>
+#include <ifunc-wcscmp.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -528,6 +529,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCSNCAT_IFUNC */

+#if HAVE_WCSCMP_IFUNC
+ IFUNC_IMPL (i, name, wcscmp,
+# if HAVE_WCSCMP_Z13
+ IFUNC_IMPL_ADD (array, i, wcscmp,
+ dl_hwcap & HWCAP_S390_VX, WCSCMP_Z13)
+# endif
+# if HAVE_WCSCMP_C
+ IFUNC_IMPL_ADD (array, i, wcscmp, 1, WCSCMP_C)
+# endif
+ )
+#endif /* HAVE_WCSCMP_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -536,8 +549,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcscmp);
-
IFUNC_VX_IMPL (wcsncmp);

IFUNC_VX_IMPL (wcschr);
diff --git a/sysdeps/s390/multiarch/wcscmp-c.c b/sysdeps/s390/wcscmp-c.c
similarity index 72%
rename from sysdeps/s390/multiarch/wcscmp-c.c
rename to sysdeps/s390/wcscmp-c.c
index ce0817ae97..643ba7a682 100644
--- a/sysdeps/s390/multiarch/wcscmp-c.c
+++ b/sysdeps/s390/wcscmp-c.c
@@ -16,17 +16,19 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSCMP __wcscmp_c
+#include <ifunc-wcscmp.h>

-# include <wchar.h>
-extern __typeof (wcscmp) __wcscmp_c;
-# undef weak_alias
-# define weak_alias(name, alias)
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
+#if HAVE_WCSCMP_C
+# if HAVE_WCSCMP_IFUNC
+# define WCSCMP WCSCMP_C
+# undef weak_alias
+# define weak_alias(name, alias)
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_def
+# define libc_hidden_def(name) \
__hidden_ver1 (__wcscmp_c, __GI___wcscmp, __wcscmp_c);
-# endif /* SHARED */
+# endif
+# endif
+
# include <wcsmbs/wcscmp.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/wcscmp-vx.S b/sysdeps/s390/wcscmp-vx.S
similarity index 92%
rename from sysdeps/s390/multiarch/wcscmp-vx.S
rename to sysdeps/s390/wcscmp-vx.S
index 14267dbfc7..a2789d8a69 100644
--- a/sysdeps/s390/multiarch/wcscmp-vx.S
+++ b/sysdeps/s390/wcscmp-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcscmp.h>
+#if HAVE_WCSCMP_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -36,7 +37,7 @@
-v17=part of s2
-v18=index of unequal
*/
-ENTRY(__wcscmp_vx)
+ENTRY(WCSCMP_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -127,5 +128,14 @@ ENTRY(__wcscmp_vx)
.Lend_equal:
lghi %r2,0
br %r14
-END(__wcscmp_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(WCSCMP_Z13)
+
+# if ! HAVE_WCSCMP_IFUNC
+strong_alias (WCSCMP_Z13, __wcscmp)
+weak_alias (__wcscmp, wcscmp)
+# endif
+
+# if ! HAVE_WCSCMP_C && defined SHARED && IS_IN (libc)
+strong_alias (WCSCMP_Z13, __GI___wcscmp)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcscmp.c b/sysdeps/s390/wcscmp.c
similarity index 70%
rename from sysdeps/s390/multiarch/wcscmp.c
rename to sysdeps/s390/wcscmp.c
index 5ee0fd4d88..769c73506e 100644
--- a/sysdeps/s390/multiarch/wcscmp.c
+++ b/sysdeps/s390/wcscmp.c
@@ -16,15 +16,26 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcscmp.h>
+
+#if HAVE_WCSCMP_IFUNC
# define __wcscmp __redirect___wcscmp
# include <wchar.h>
# undef __wcscmp
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc_redirected (__redirect___wcscmp, __wcscmp)
-weak_alias (__wcscmp, wcscmp)
+# if HAVE_WCSCMP_C
+extern __typeof (__redirect___wcscmp) WCSCMP_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcscmp.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCSCMP_Z13
+extern __typeof (__redirect___wcscmp) WCSCMP_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect___wcscmp, __wcscmp,
+ (HAVE_WCSCMP_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSCMP_Z13
+ : WCSCMP_DEFAULT
+ )
+weak_alias (__wcscmp, wcscmp)
+#endif

[Stefan Liebler]

The ifunc handling for wmemcmp is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wmemcmp variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wmemcmp variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wmemcmp.
* sysdeps/s390/multiarch/wmemcmp-c.c: Move to ...
* sysdeps/s390/wmemcmp-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wmemcmp-vx.S: Move to ...
* sysdeps/s390/wmemcmp-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wmemcmp.c: Move to ...
* sysdeps/s390/wmemcmp.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wmemcmp.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wmemcmp.h | 52 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 4 --
sysdeps/s390/multiarch/ifunc-impl-list.c | 23 +++++-----
sysdeps/s390/{multiarch => }/wmemcmp-c.c | 9 ++--
sysdeps/s390/{multiarch => }/wmemcmp-vx.S | 13 ++++--
sysdeps/s390/{multiarch => }/wmemcmp.c | 21 ++++++---
7 files changed, 96 insertions(+), 29 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wmemcmp.h
rename sysdeps/s390/{multiarch => }/wmemcmp-c.c (85%)
rename sysdeps/s390/{multiarch => }/wmemcmp-vx.S (95%)
rename sysdeps/s390/{multiarch => }/wmemcmp.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index ae2ae3d2ea..35fcf7e21d 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -100,5 +100,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcspbrk wcspbrk-vx wcspbrk-c \
wcscspn wcscspn-vx wcscspn-c \
wmemchr wmemchr-vx wmemchr-c \
- wmemset wmemset-vx wmemset-c
+ wmemset wmemset-vx wmemset-c \
+ wmemcmp wmemcmp-vx wmemcmp-c
endif
diff --git a/sysdeps/s390/ifunc-wmemcmp.h b/sysdeps/s390/ifunc-wmemcmp.h
new file mode 100644
index 0000000000..1b38a01459
--- /dev/null
+++ b/sysdeps/s390/ifunc-wmemcmp.h
@@ -0,0 +1,52 @@
+/* wmemcmp variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WMEMCMP_IFUNC 1
+#else
+# define HAVE_WMEMCMP_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WMEMCMP_IFUNC_AND_VX_SUPPORT HAVE_WMEMCMP_IFUNC
+#else
+# define HAVE_WMEMCMP_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WMEMCMP_DEFAULT WMEMCMP_Z13
+# define HAVE_WMEMCMP_C 0
+# define HAVE_WMEMCMP_Z13 1
+#else
+# define WMEMCMP_DEFAULT WMEMCMP_C
+# define HAVE_WMEMCMP_C 1
+# define HAVE_WMEMCMP_Z13 HAVE_WMEMCMP_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WMEMCMP_C
+# define WMEMCMP_C __wmemcmp_c
+#else
+# define WMEMCMP_C NULL
+#endif
+
+#if HAVE_WMEMCMP_Z13
+# define WMEMCMP_Z13 __wmemcmp_vx
+#else
+# define WMEMCMP_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index cc6dd7adb1..fec3615304 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,7 +1,3 @@
-ifeq ($(subdir),wcsmbs)
-sysdep_routines += wmemcmp wmemcmp-vx wmemcmp-c
-endif
-
ifeq ($(subdir),iconvdata)
override define generate-8bit-table
$(make-target-directory)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 7040959269..177c5fd6fe 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -64,6 +64,7 @@
#include <ifunc-wcscspn.h>
#include <ifunc-wmemchr.h>
#include <ifunc-wmemset.h>
+#include <ifunc-wmemcmp.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -658,17 +659,17 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WMEMSET_IFUNC */

-#ifdef HAVE_S390_VX_ASM_SUPPORT
-
-# define IFUNC_VX_IMPL(FUNC) \
- IFUNC_IMPL (i, name, FUNC, \
- IFUNC_IMPL_ADD (array, i, FUNC, dl_hwcap & HWCAP_S390_VX, \
- __##FUNC##_vx) \
- IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))
-
- IFUNC_VX_IMPL (wmemcmp);
-
-#endif /* HAVE_S390_VX_ASM_SUPPORT */
+#if HAVE_WMEMCMP_IFUNC
+ IFUNC_IMPL (i, name, wmemcmp,
+# if HAVE_WMEMCMP_Z13
+ IFUNC_IMPL_ADD (array, i, wmemcmp,
+ dl_hwcap & HWCAP_S390_VX, WMEMCMP_Z13)
+# endif
+# if HAVE_WMEMCMP_C
+ IFUNC_IMPL_ADD (array, i, wmemcmp, 1, WMEMCMP_C)
+# endif
+ )
+#endif /* HAVE_WMEMCMP_IFUNC */

return i;
}
diff --git a/sysdeps/s390/multiarch/wmemcmp-c.c b/sysdeps/s390/wmemcmp-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/wmemcmp-c.c
rename to sysdeps/s390/wmemcmp-c.c
index 2fd39d5013..0c73636add 100644
--- a/sysdeps/s390/multiarch/wmemcmp-c.c
+++ b/sysdeps/s390/wmemcmp-c.c
@@ -16,11 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WMEMCMP __wmemcmp_c
+#include <ifunc-wmemcmp.h>

-# include <wchar.h>
-extern __typeof (wmemcmp) __wmemcmp_c;
+#if HAVE_WMEMCMP_C
+# if HAVE_WMEMCMP_IFUNC
+# define WMEMCMP WMEMCMP_C
+# endif

# include <wcsmbs/wmemcmp.c>
#endif
diff --git a/sysdeps/s390/multiarch/wmemcmp-vx.S b/sysdeps/s390/wmemcmp-vx.S
similarity index 95%
rename from sysdeps/s390/multiarch/wmemcmp-vx.S
rename to sysdeps/s390/wmemcmp-vx.S
index e2fc21e419..87ae21b4f1 100644
--- a/sysdeps/s390/multiarch/wmemcmp-vx.S
+++ b/sysdeps/s390/wmemcmp-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wmemcmp.h>
+#if HAVE_WMEMCMP_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -37,7 +38,7 @@
-v17=part of s2
-v18=index of unequal
*/
-ENTRY(__wmemcmp_vx)
+ENTRY(WMEMCMP_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -145,5 +146,9 @@ ENTRY(__wmemcmp_vx)
la %r2,0(%r5,%r2)
la %r3,0(%r5,%r3)
j .Lremaining
-END(__wmemcmp_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(WMEMCMP_Z13)
+
+# if ! HAVE_WMEMCMP_IFUNC
+strong_alias (WMEMCMP_Z13, wmemcmp)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wmemcmp.c b/sysdeps/s390/wmemcmp.c
similarity index 69%
rename from sysdeps/s390/multiarch/wmemcmp.c
rename to sysdeps/s390/wmemcmp.c
index a4cb440c45..6c8ca5f0e8 100644
--- a/sysdeps/s390/multiarch/wmemcmp.c
+++ b/sysdeps/s390/wmemcmp.c
@@ -16,12 +16,23 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wmemcmp.h>
+
+#if HAVE_WMEMCMP_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2 (__wmemcmp, wmemcmp)
+# if HAVE_WMEMCMP_C
+extern __typeof (wmemcmp) WMEMCMP_C attribute_hidden;
+# endif
+
+# if HAVE_WMEMCMP_Z13
+extern __typeof (wmemcmp) WMEMCMP_Z13 attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wmemcmp.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (wmemcmp, wmemcmp,
+ (HAVE_WMEMCMP_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WMEMCMP_Z13
+ : WMEMCMP_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for strlen is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strlen variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strlen variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strlen.
* sysdeps/s390/multiarch/strlen-c.c: Move to ...
* sysdeps/s390/strlen-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strlen-vx.S: Move to ...
* sysdeps/s390/strlen-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strlen.c: Move to ...
* sysdeps/s390/strlen.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strlen.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strlen.h | 52 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 ++++++-
sysdeps/s390/{multiarch => }/strlen-c.c | 18 ++++----
sysdeps/s390/{multiarch => }/strlen-vx.S | 17 ++++++--
sysdeps/s390/{multiarch => }/strlen.c | 21 +++++++---
7 files changed, 108 insertions(+), 20 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strlen.h
rename sysdeps/s390/{multiarch => }/strlen-c.c (78%)
rename sysdeps/s390/{multiarch => }/strlen-vx.S (90%)
rename sysdeps/s390/{multiarch => }/strlen.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 0a964e0d4d..bc21b06825 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -59,5 +59,6 @@ sysdep_routines += bzero memset memset-z900 \
mempcpy memcpy memcpy-z900 \
memmove memmove-c \
strstr strstr-vx strstr-c \
- memmem memmem-vx memmem-c
+ memmem memmem-vx memmem-c \
+ strlen strlen-vx strlen-c
endif
diff --git a/sysdeps/s390/ifunc-strlen.h b/sysdeps/s390/ifunc-strlen.h
new file mode 100644
index 0000000000..f207059663
--- /dev/null
+++ b/sysdeps/s390/ifunc-strlen.h
@@ -0,0 +1,52 @@
+/* strlen variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRLEN_IFUNC 1
+#else
+# define HAVE_STRLEN_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRLEN_IFUNC_AND_VX_SUPPORT HAVE_STRLEN_IFUNC
+#else
+# define HAVE_STRLEN_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRLEN_DEFAULT STRLEN_Z13
+# define HAVE_STRLEN_C 0
+# define HAVE_STRLEN_Z13 1
+#else
+# define STRLEN_DEFAULT STRLEN_C
+# define HAVE_STRLEN_C 1
+# define HAVE_STRLEN_Z13 HAVE_STRLEN_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRLEN_C
+# define STRLEN_C __strlen_c
+#else
+# define STRLEN_C NULL
+#endif
+
+#if HAVE_STRLEN_Z13
+# define STRLEN_Z13 __strlen_vx
+#else
+# define STRLEN_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 24949cd3a8..601523919c 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strlen strlen-vx strlen-c \
- strnlen strnlen-vx strnlen-c \
+sysdep_routines += strnlen strnlen-vx strnlen-c \
strcpy strcpy-vx \
stpcpy stpcpy-vx stpcpy-c \
strncpy strncpy-vx \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index da8696d917..c531be4bc7 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -26,6 +26,7 @@
#include <ifunc-memcpy.h>
#include <ifunc-strstr.h>
#include <ifunc-memmem.h>
+#include <ifunc-strlen.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -164,6 +165,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_MEMMEM_IFUNC */

+#if HAVE_STRLEN_IFUNC
+ IFUNC_IMPL (i, name, strlen,
+# if HAVE_STRLEN_Z13
+ IFUNC_IMPL_ADD (array, i, strlen,
+ dl_hwcap & HWCAP_S390_VX, STRLEN_Z13)
+# endif
+# if HAVE_STRLEN_C
+ IFUNC_IMPL_ADD (array, i, strlen, 1, STRLEN_C)
+# endif
+ )
+#endif /* HAVE_STRLEN_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -172,7 +185,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (strlen);
IFUNC_VX_IMPL (wcslen);

IFUNC_VX_IMPL (strnlen);
diff --git a/sysdeps/s390/multiarch/strlen-c.c b/sysdeps/s390/strlen-c.c
similarity index 78%
rename from sysdeps/s390/multiarch/strlen-c.c
rename to sysdeps/s390/strlen-c.c
index a2c8e43624..b4569701af 100644
--- a/sysdeps/s390/multiarch/strlen-c.c
+++ b/sysdeps/s390/strlen-c.c
@@ -16,13 +16,17 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRLEN __strlen_c
-# ifdef SHARED
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
+#include <ifunc-strlen.h>
+
+#if HAVE_STRLEN_C
+# if HAVE_STRLEN_IFUNC
+# define STRLEN STRLEN_C
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
__hidden_ver1 (__strlen_c, __GI_strlen, __strlen_c);
-# endif /* SHARED */
+# endif
+# endif

# include <string/strlen.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strlen-vx.S b/sysdeps/s390/strlen-vx.S
similarity index 90%
rename from sysdeps/s390/multiarch/strlen-vx.S
rename to sysdeps/s390/strlen-vx.S
index 9308b33237..39ef43107d 100644
--- a/sysdeps/s390/multiarch/strlen-vx.S
+++ b/sysdeps/s390/strlen-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strlen.h>
+#if HAVE_STRLEN_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -34,7 +35,7 @@
-r5=current_len and return_value
-v16=part of s
*/
-ENTRY(__strlen_vx)
+ENTRY(STRLEN_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -80,5 +81,13 @@ ENTRY(__strlen_vx)
vlgvb %r2,%v16,7 /* Load byte index of zero. */
algr %r2,%r5
br %r14
-END(__strlen_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRLEN_Z13)
+
+# if ! HAVE_STRLEN_IFUNC
+strong_alias (STRLEN_Z13, strlen)
+# endif
+
+# if ! HAVE_STRLEN_C && defined SHARED && IS_IN (libc)
+strong_alias (STRLEN_Z13, __GI_strlen)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/strlen.c b/sysdeps/s390/strlen.c
similarity index 69%
rename from sysdeps/s390/multiarch/strlen.c
rename to sysdeps/s390/strlen.c
index 0edf8b7d02..6ba0fe86fe 100644
--- a/sysdeps/s390/multiarch/strlen.c
+++ b/sysdeps/s390/strlen.c
@@ -16,14 +16,25 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strlen.h>
+
+#if HAVE_STRLEN_IFUNC
# define strlen __redirect_strlen
# include <string.h>
# include <ifunc-resolve.h>
# undef strlen

-s390_vx_libc_ifunc2_redirected (__redirect_strlen, __strlen, strlen)
+# if HAVE_STRLEN_C
+extern __typeof (__redirect_strlen) STRLEN_C attribute_hidden;
+# endif
+
+# if HAVE_STRLEN_Z13
+extern __typeof (__redirect_strlen) STRLEN_Z13 attribute_hidden;
+# endif

-#else
-# include <string/strlen.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_strlen, strlen,
+ (HAVE_STRLEN_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRLEN_Z13
+ : STRLEN_DEFAULT
+ )
+#endif

[Stefan Liebler]

The ifunc handling for wcscat is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcscat variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcscat variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcscat.
* sysdeps/s390/multiarch/wcscat-c.c: Move to ...
* sysdeps/s390/wcscat-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcscat-vx.S: Move to ...
* sysdeps/s390/wcscat-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcscat.c: Move to ...
* sysdeps/s390/wcscat.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcscat.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcscat.h | 53 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcscat-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcscat-vx.S | 16 ++++---
sysdeps/s390/{multiarch => }/wcscat.c | 23 +++++++---
7 files changed, 103 insertions(+), 20 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcscat.h
rename sysdeps/s390/{multiarch => }/wcscat-c.c (86%)
rename sysdeps/s390/{multiarch => }/wcscat-vx.S (96%)
rename sysdeps/s390/{multiarch => }/wcscat.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index e49b1bd26b..f393a2b1ef 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -88,5 +88,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcscpy wcscpy-vx wcscpy-c \
wcpcpy wcpcpy-vx wcpcpy-c \
wcsncpy wcsncpy-vx wcsncpy-c \
- wcpncpy wcpncpy-vx wcpncpy-c
+ wcpncpy wcpncpy-vx wcpncpy-c \
+ wcscat wcscat-vx wcscat-c
endif
diff --git a/sysdeps/s390/ifunc-wcscat.h b/sysdeps/s390/ifunc-wcscat.h
new file mode 100644
index 0000000000..fecae21403
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcscat.h
@@ -0,0 +1,53 @@
+/* wcscat variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSCAT_IFUNC 1
+#else
+# define HAVE_WCSCAT_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSCAT_IFUNC_AND_VX_SUPPORT HAVE_WCSCAT_IFUNC
+#else
+# define HAVE_WCSCAT_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSCAT_DEFAULT WCSCAT_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSCAT_C 1
+# define HAVE_WCSCAT_Z13 1
+#else
+# define WCSCAT_DEFAULT WCSCAT_C
+# define HAVE_WCSCAT_C 1
+# define HAVE_WCSCAT_Z13 HAVE_WCSCAT_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSCAT_C
+# define WCSCAT_C __wcscat_c
+#else
+# define WCSCAT_C NULL
+#endif
+
+#if HAVE_WCSCAT_Z13
+# define WCSCAT_Z13 __wcscat_vx
+#else
+# define WCSCAT_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 158fb49552..617017496c 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcscat wcscat-vx wcscat-c \
- wcsncat wcsncat-vx wcsncat-c \
+sysdep_routines += wcsncat wcsncat-vx wcsncat-c \
wcscmp wcscmp-vx wcscmp-c \
wcsncmp wcsncmp-vx wcsncmp-c \
wcschr wcschr-vx wcschr-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index e60238fcde..b05bd35fd8 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -52,6 +52,7 @@
#include <ifunc-wcpcpy.h>
#include <ifunc-wcsncpy.h>
#include <ifunc-wcpncpy.h>
+#include <ifunc-wcscat.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -502,6 +503,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCPNCPY_IFUNC */

+#if HAVE_WCSCAT_IFUNC
+ IFUNC_IMPL (i, name, wcscat,
+# if HAVE_WCSCAT_Z13
+ IFUNC_IMPL_ADD (array, i, wcscat,
+ dl_hwcap & HWCAP_S390_VX, WCSCAT_Z13)
+# endif
+# if HAVE_WCSCAT_C
+ IFUNC_IMPL_ADD (array, i, wcscat, 1, WCSCAT_C)
+# endif
+ )
+#endif /* HAVE_WCSCAT_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -510,8 +523,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcscat);
-
IFUNC_VX_IMPL (wcsncat);

IFUNC_VX_IMPL (wcscmp);
diff --git a/sysdeps/s390/multiarch/wcscat-c.c b/sysdeps/s390/wcscat-c.c
similarity index 86%
rename from sysdeps/s390/multiarch/wcscat-c.c
rename to sysdeps/s390/wcscat-c.c
index 9a31c65a0b..bc1a50b1a5 100644
--- a/sysdeps/s390/multiarch/wcscat-c.c
+++ b/sysdeps/s390/wcscat-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSCAT __wcscat_c
+#include <ifunc-wcscat.h>
+
+#if HAVE_WCSCAT_C
+# if HAVE_WCSCAT_IFUNC || HAVE_WCSCAT_Z13
+# define WCSCAT WCSCAT_C
+# endif

-# include <wchar.h>
-extern __typeof (__wcscat) __wcscat_c;
# include <wcsmbs/wcscat.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcscat-vx.S b/sysdeps/s390/wcscat-vx.S
similarity index 96%
rename from sysdeps/s390/multiarch/wcscat-vx.S
rename to sysdeps/s390/wcscat-vx.S
index 2164a8da41..4e40d69e84 100644
--- a/sysdeps/s390/multiarch/wcscat-vx.S
+++ b/sysdeps/s390/wcscat-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcscat.h>
+#if HAVE_WCSCAT_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -37,7 +38,7 @@
-v17=index of zero
-v18=part of src
*/
-ENTRY(__wcscat_vx)
+ENTRY(WCSCAT_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -170,6 +171,11 @@ ENTRY(__wcscat_vx)
lgr %r2,%r0 /* Load saved dest-ptr. */
br %r14
.Lfallback:
- jg __wcscat_c
-END(__wcscat_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSCAT_C
+END(WCSCAT_Z13)
+
+# if ! HAVE_WCSCAT_IFUNC
+strong_alias (WCSCAT_Z13, __wcscat)
+weak_alias (__wcscat, wcscat)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcscat.c b/sysdeps/s390/wcscat.c
similarity index 70%
rename from sysdeps/s390/multiarch/wcscat.c
rename to sysdeps/s390/wcscat.c
index 33e4f6da3f..3741210a17 100644
--- a/sysdeps/s390/multiarch/wcscat.c
+++ b/sysdeps/s390/wcscat.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcscat.h>
+
+#if HAVE_WCSCAT_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__wcscat)
-weak_alias (__wcscat, wcscat)
+# if HAVE_WCSCAT_C
+extern __typeof (__wcscat) WCSCAT_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcscat.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCSCAT_Z13
+extern __typeof (__wcscat) WCSCAT_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__wcscat, __wcscat,
+ (HAVE_WCSCAT_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSCAT_Z13
+ : WCSCAT_DEFAULT
+ )
+weak_alias (__wcscat, wcscat)
+#endif

[Stefan Liebler]

This patch moves all ifunc variants for memcpy/mempcpy
to sysdeps/s390/memcpy-z900.S. The configure-check/preprocessor logic
in sysdeps/s390/ifunc-memcpy.h decides if ifunc is needed at all
and which ifunc variants should be available.
E.g. if the compiler/assembler already supports z196 by default,
the older ifunc variants are not included.
If we only need the newest ifunc variant,
then we can skip ifunc at all.

Therefore the ifunc-resolvers and __libc_ifunc_impl_list are adjusted
in order to handle only the available ifunc variants.

ChangeLog:

* sysdeps/s390/ifunc-memcpy.h: New File.
* sysdeps/s390/memcpy.S: Move to ...
* sysdeps/s390/memcpy-z900.S ... here.
Move implementations from memcpy-s390x.s to here.
* sysdeps/s390/multiarch/memcpy-s390x.S: Delete File.
* sysdeps/s390/multiarch/Makefile (sysdep_routines):
Remove memcpy/mempcpy variants.
* sysdeps/s390/Makefile (sysdep_routines):
Add memcpy/mempcpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Adjust ifunc variants for
memcpy and mempcpy.
* sysdeps/s390/multiarch/memcpy.c: Move ifunc resolver
to ...
* sysdeps/s390/memcpy.c: ... here.
Adjust ifunc variants for memcpy.
* sysdeps/s390/multiarch/mempcpy.c: Move to ...
* sysdeps/s390/mempcpy.c: ... here.
Adjust ifunc variants for mempcpy.
* sysdeps/s390/mempcpy.S: Delete file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-memcpy.h | 68 +++++++++++
sysdeps/s390/{memcpy.S => memcpy-z900.S} | 148 +++++++++++++++++++----
sysdeps/s390/{multiarch => }/memcpy.c | 28 ++++-
sysdeps/s390/mempcpy.S | 19 ---
sysdeps/s390/{multiarch => }/mempcpy.c | 26 +++-
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 33 +++--
sysdeps/s390/multiarch/memcpy-s390x.S | 132 --------------------
9 files changed, 265 insertions(+), 195 deletions(-)
create mode 100644 sysdeps/s390/ifunc-memcpy.h
rename sysdeps/s390/{memcpy.S => memcpy-z900.S} (50%)
rename sysdeps/s390/{multiarch => }/memcpy.c (60%)
delete mode 100644 sysdeps/s390/mempcpy.S
rename sysdeps/s390/{multiarch => }/mempcpy.c (63%)
delete mode 100644 sysdeps/s390/multiarch/memcpy-s390x.S

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 44a2f28bfd..c196a19bfb 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -55,5 +55,6 @@ endif

ifeq ($(subdir),string)
sysdep_routines += bzero memset memset-z900 \
- memcmp memcmp-z900
+ memcmp memcmp-z900 \
+ mempcpy memcpy memcpy-z900
endif
diff --git a/sysdeps/s390/ifunc-memcpy.h b/sysdeps/s390/ifunc-memcpy.h
new file mode 100644
index 0000000000..51c71baa2c
--- /dev/null
+++ b/sysdeps/s390/ifunc-memcpy.h
@@ -0,0 +1,68 @@
+/* memcpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined SHARED && defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
+# define HAVE_MEMCPY_IFUNC 1
+#else
+# define HAVE_MEMCPY_IFUNC 0
+#endif
+
+#if defined HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
+# define MEMCPY_DEFAULT MEMCPY_Z196
+# define MEMPCPY_DEFAULT MEMPCPY_Z196
+# define HAVE_MEMCPY_Z900_G5 0
+# define HAVE_MEMCPY_Z10 0
+# define HAVE_MEMCPY_Z196 1
+#elif defined HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
+# define MEMCPY_DEFAULT MEMCPY_Z10
+# define MEMPCPY_DEFAULT MEMPCPY_Z10
+# define HAVE_MEMCPY_Z900_G5 0
+# define HAVE_MEMCPY_Z10 1
+# define HAVE_MEMCPY_Z196 HAVE_MEMCPY_IFUNC
+#else
+# define MEMCPY_DEFAULT MEMCPY_Z900_G5
+# define MEMPCPY_DEFAULT MEMPCPY_Z900_G5
+# define HAVE_MEMCPY_Z900_G5 1
+# define HAVE_MEMCPY_Z10 HAVE_MEMCPY_IFUNC
+# define HAVE_MEMCPY_Z196 HAVE_MEMCPY_IFUNC
+#endif
+
+#if HAVE_MEMCPY_Z900_G5
+# define MEMCPY_Z900_G5 __memcpy_default
+# define MEMPCPY_Z900_G5 __mempcpy_default
+#else
+# define MEMCPY_Z900_G5 NULL
+# define MEMPCPY_Z900_G5 NULL
+#endif
+
+#if HAVE_MEMCPY_Z10
+# define MEMCPY_Z10 __memcpy_z10
+# define MEMPCPY_Z10 __mempcpy_z10
+#else
+# define MEMCPY_Z10 NULL
+# define MEMPCPY_Z10 NULL
+#endif
+
+#if HAVE_MEMCPY_Z196
+# define MEMCPY_Z196 __memcpy_z196
+# define MEMPCPY_Z196 __mempcpy_z196
+#else
+# define MEMCPY_Z196 NULL
+# define MEMPCPY_Z196 NULL
+#endif
diff --git a/sysdeps/s390/memcpy.S b/sysdeps/s390/memcpy-z900.S
similarity index 50%
rename from sysdeps/s390/memcpy.S
rename to sysdeps/s390/memcpy-z900.S
index 2a6c6b7503..3a50cf44d8 100644
--- a/sysdeps/s390/memcpy.S
+++ b/sysdeps/s390/memcpy-z900.S
@@ -19,6 +19,7 @@

#include <sysdep.h>
#include "asm-syntax.h"
+#include <ifunc-memcpy.h>

/* INPUT PARAMETERS
%r2 = address of destination memory area
@@ -41,50 +42,46 @@
# define BRCTG brct
#endif /* ! defined __s390x__ */

-ENTRY(__mempcpy)
-#if defined __s390x__
+#if HAVE_MEMCPY_Z900_G5
+ENTRY(MEMPCPY_Z900_G5)
+# if defined __s390x__
.machine "z900"
-#else
+# else
.machine "g5"
-#endif /* ! defined __s390x__ */
+# endif /* ! defined __s390x__ */
LGR %r1,%r2 # Use as dest
la %r2,0(%r4,%r2) # Return dest + n
j .L_Z900_G5_start
-END(__mempcpy)
-#ifndef USE_MULTIARCH
-libc_hidden_def (__mempcpy)
-weak_alias (__mempcpy, mempcpy)
-libc_hidden_builtin_def (mempcpy)
-#endif
+END(MEMPCPY_Z900_G5)

-ENTRY(memcpy)
-#if defined __s390x__
+ENTRY(MEMCPY_Z900_G5)
+# if defined __s390x__
.machine "z900"
-#else
+# else
.machine "g5"
-#endif /* ! defined __s390x__ */
+# endif /* ! defined __s390x__ */
LGR %r1,%r2 # r1: Use as dest ; r2: Return dest
.L_Z900_G5_start:
LTGR %r4,%r4
je .L_Z900_G5_4
AGHI %r4,-1
-#if defined __s390x__
+# if defined __s390x__
srlg %r5,%r4,8
-#else
+# else
lr %r5,%r4
srl %r5,8
-#endif /* ! defined __s390x__ */
+# endif /* ! defined __s390x__ */
LTGR %r5,%r5
jne .L_Z900_G5_13
.L_Z900_G5_3:
-#if defined __s390x__
+# if defined __s390x__
larl %r5,.L_Z900_G5_15
-# define Z900_G5_EX_D 0
-#else
+# define Z900_G5_EX_D 0
+# else
basr %r5,0
.L_Z900_G5_14:
-# define Z900_G5_EX_D .L_Z900_G5_15-.L_Z900_G5_14
-#endif /* ! defined __s390x__ */
+# define Z900_G5_EX_D .L_Z900_G5_15-.L_Z900_G5_14
+# endif /* ! defined __s390x__ */
ex %r4,Z900_G5_EX_D(%r5)
.L_Z900_G5_4:
br %r14
@@ -99,10 +96,8 @@ ENTRY(memcpy)
j .L_Z900_G5_3
.L_Z900_G5_15:
mvc 0(1,%r1),0(%r3)
-END(memcpy)
-#ifndef USE_MULTIARCH
-libc_hidden_builtin_def (memcpy)
-#endif
+END(MEMCPY_Z900_G5)
+#endif /* HAVE_MEMCPY_Z900_G5 */

ENTRY(__memcpy_mvcle)
# Using as standalone function will result in unexpected
@@ -126,3 +121,104 @@ END(__memcpy_mvcle)
#undef LGR
#undef AGHI
#undef BRCTG
+
+#if HAVE_MEMCPY_Z10
+ENTRY(MEMPCPY_Z10)
+ .machine "z10"
+ .machinemode "zarch_nohighgprs"
+ lgr %r1,%r2 # Use as dest
+ la %r2,0(%r4,%r2) # Return dest + n
+ j .L_Z10_start
+END(MEMPCPY_Z10)
+
+ENTRY(MEMCPY_Z10)
+ .machine "z10"
+ .machinemode "zarch_nohighgprs"
+ lgr %r1,%r2 # r1: Use as dest ; r2: Return dest
+.L_Z10_start:
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+ cgije %r4,0,.L_Z10_4
+ aghi %r4,-1
+ srlg %r5,%r4,8
+ cgijlh %r5,0,.L_Z10_13
+.L_Z10_3:
+ exrl %r4,.L_Z10_15
+.L_Z10_4:
+ br %r14
+.L_Z10_13:
+ cgfi %r5,65535 # Switch to mvcle for copies >16MB
+ jh __memcpy_mvcle
+.L_Z10_12:
+ pfd 1,768(%r3)
+ pfd 2,768(%r1)
+ mvc 0(256,%r1),0(%r3)
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ brctg %r5,.L_Z10_12
+ j .L_Z10_3
+.L_Z10_15:
+ mvc 0(1,%r1),0(%r3)
+END(MEMCPY_Z10)
+#endif /* HAVE_MEMCPY_Z10 */
+
+#if HAVE_MEMCPY_Z196
+ENTRY(MEMPCPY_Z196)
+ .machine "z196"
+ .machinemode "zarch_nohighgprs"
+ lgr %r1,%r2 # Use as dest
+ la %r2,0(%r4,%r2) # Return dest + n
+ j .L_Z196_start
+END(MEMPCPY_Z196)
+
+ENTRY(MEMCPY_Z196)
+ .machine "z196"
+ .machinemode "zarch_nohighgprs"
+ lgr %r1,%r2 # r1: Use as dest ; r2: Return dest
+.L_Z196_start:
+# if !defined __s390x__
+ llgfr %r4,%r4
+# endif /* !defined __s390x__ */
+ ltgr %r4,%r4
+ je .L_Z196_4
+ aghi %r4,-1
+ srlg %r5,%r4,8
+ ltgr %r5,%r5
+ jne .L_Z196_5
+.L_Z196_3:
+ exrl %r4,.L_Z196_14
+.L_Z196_4:
+ br %r14
+.L_Z196_5:
+ cgfi %r5,262144 # Switch to mvcle for copies >64MB
+ jh __memcpy_mvcle
+.L_Z196_2:
+ pfd 1,768(%r3)
+ pfd 2,768(%r1)
+ mvc 0(256,%r1),0(%r3)
+ aghi %r5,-1
+ la %r1,256(%r1)
+ la %r3,256(%r3)
+ jne .L_Z196_2
+ j .L_Z196_3
+.L_Z196_14:
+ mvc 0(1,%r1),0(%r3)
+END(MEMCPY_Z196)
+#endif /* HAVE_MEMCPY_Z196 */
+
+#if ! HAVE_MEMCPY_IFUNC
+/* If we don't use ifunc, define an alias for mem[p]cpy here.
+ Otherwise see sysdeps/s390/mem[p]cpy.c. */
+strong_alias (MEMCPY_DEFAULT, memcpy)
+strong_alias (MEMPCPY_DEFAULT, __mempcpy)
+weak_alias (__mempcpy, mempcpy)
+#endif
+
+#if defined SHARED && IS_IN (libc)
+/* Defines the internal symbols.
+ Compare to libc_hidden_[builtin_]def (mem[p]cpy) in string/mem[p]cpy.c. */
+strong_alias (MEMCPY_DEFAULT, __GI_memcpy)
+strong_alias (MEMPCPY_DEFAULT, __GI_mempcpy)
+strong_alias (MEMPCPY_DEFAULT, __GI___mempcpy)
+#endif
diff --git a/sysdeps/s390/multiarch/memcpy.c b/sysdeps/s390/memcpy.c
similarity index 60%
rename from sysdeps/s390/multiarch/memcpy.c
rename to sysdeps/s390/memcpy.c
index c9577a854a..90a53ac27d 100644
--- a/sysdeps/s390/multiarch/memcpy.c
+++ b/sysdeps/s390/memcpy.c
@@ -16,12 +16,34 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-/* In the static lib memcpy is needed before the reloc is resolved. */
-#if defined SHARED && IS_IN (libc)
+#include <ifunc-memcpy.h>
+
+#if HAVE_MEMCPY_IFUNC
# define memcpy __redirect_memcpy
# include <string.h>
# undef memcpy
# include <ifunc-resolve.h>

-s390_libc_ifunc (__redirect_memcpy, __memcpy, memcpy)
+# if HAVE_MEMCPY_Z900_G5
+extern __typeof (__redirect_memcpy) MEMCPY_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_MEMCPY_Z10
+extern __typeof (__redirect_memcpy) MEMCPY_Z10 attribute_hidden;
+# endif
+
+# if HAVE_MEMCPY_Z196
+extern __typeof (__redirect_memcpy) MEMCPY_Z196 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect_memcpy, memcpy,
+ ({
+ s390_libc_ifunc_init ();
+ (HAVE_MEMCPY_Z196 && S390_IS_Z196 (stfle_bits))
+ ? MEMCPY_Z196
+ : (HAVE_MEMCPY_Z10 && S390_IS_Z10 (stfle_bits))
+ ? MEMCPY_Z10
+ : MEMCPY_DEFAULT;
+ })
+ )
#endif
diff --git a/sysdeps/s390/mempcpy.S b/sysdeps/s390/mempcpy.S
deleted file mode 100644
index 18ef29213e..0000000000
--- a/sysdeps/s390/mempcpy.S
+++ /dev/null
@@ -1,19 +0,0 @@
-/* CPU specific mempcpy without multiarch - 32/64 bit S/390 version.
- Copyright (C) 2016-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* mempcpy is implemented in memcpy.S. */
diff --git a/sysdeps/s390/multiarch/mempcpy.c b/sysdeps/s390/mempcpy.c
similarity index 63%
rename from sysdeps/s390/multiarch/mempcpy.c
rename to sysdeps/s390/mempcpy.c
index 363fe47aef..a6a2373126 100644
--- a/sysdeps/s390/multiarch/mempcpy.c
+++ b/sysdeps/s390/mempcpy.c
@@ -16,8 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

+#include <ifunc-memcpy.h>

-#if defined SHARED && IS_IN (libc)
+#if HAVE_MEMCPY_IFUNC
# define mempcpy __redirect_mempcpy
# define __mempcpy __redirect___mempcpy
# define __NO_STRING_INLINES
@@ -27,6 +28,27 @@
# undef __mempcpy
# include <ifunc-resolve.h>

-s390_libc_ifunc (__redirect___mempcpy, ____mempcpy, __mempcpy)
+# if HAVE_MEMCPY_Z900_G5
+extern __typeof (__redirect___mempcpy) MEMPCPY_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_MEMCPY_Z10
+extern __typeof (__redirect___mempcpy) MEMPCPY_Z10 attribute_hidden;
+# endif
+
+# if HAVE_MEMCPY_Z196
+extern __typeof (__redirect___mempcpy) MEMPCPY_Z196 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect___mempcpy, __mempcpy,
+ ({
+ s390_libc_ifunc_init ();
+ (HAVE_MEMCPY_Z196 && S390_IS_Z196 (stfle_bits))
+ ? MEMPCPY_Z196
+ : (HAVE_MEMCPY_Z10 && S390_IS_Z10 (stfle_bits))
+ ? MEMPCPY_Z10
+ : MEMPCPY_DEFAULT;
+ })
+ )
weak_alias (__mempcpy, mempcpy);
#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 3cbd5fad69..24949cd3a8 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -18,8 +18,7 @@ sysdep_routines += strlen strlen-vx strlen-c \
memchr memchr-vx \
rawmemchr rawmemchr-vx rawmemchr-c \
memccpy memccpy-vx memccpy-c \
- memrchr memrchr-vx memrchr-c \
- mempcpy memcpy memcpy-s390x
+ memrchr memrchr-vx memrchr-c
endif

ifeq ($(subdir),wcsmbs)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 2e57d01abc..6969c480cc 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -23,6 +23,7 @@
#include <ifunc-resolve.h>
#include <ifunc-memset.h>
#include <ifunc-memcmp.h>
+#include <ifunc-memcpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -95,23 +96,35 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_MEMCMP_IFUNC */

-#ifdef SHARED
-
+#if HAVE_MEMCPY_IFUNC
IFUNC_IMPL (i, name, memcpy,
+# if HAVE_MEMCPY_Z196
IFUNC_IMPL_ADD (array, i, memcpy,
- S390_IS_Z196 (stfle_bits), __memcpy_z196)
+ S390_IS_Z196 (stfle_bits), MEMCPY_Z196)
+# endif
+# if HAVE_MEMCPY_Z10
IFUNC_IMPL_ADD (array, i, memcpy,
- S390_IS_Z10 (stfle_bits), __memcpy_z10)
- IFUNC_IMPL_ADD (array, i, memcpy, 1, __memcpy_default))
+ S390_IS_Z10 (stfle_bits), MEMCPY_Z10)
+# endif
+# if HAVE_MEMCPY_Z900_G5
+ IFUNC_IMPL_ADD (array, i, memcpy, 1, MEMCPY_Z900_G5)
+# endif
+ )

IFUNC_IMPL (i, name, mempcpy,
+# if HAVE_MEMCPY_Z196
IFUNC_IMPL_ADD (array, i, mempcpy,
- S390_IS_Z196 (stfle_bits), ____mempcpy_z196)
+ S390_IS_Z196 (stfle_bits), MEMPCPY_Z196)
+# endif
+# if HAVE_MEMCPY_Z10
IFUNC_IMPL_ADD (array, i, mempcpy,
- S390_IS_Z10 (stfle_bits), ____mempcpy_z10)
- IFUNC_IMPL_ADD (array, i, mempcpy, 1, ____mempcpy_default))
-
-#endif /* SHARED */
+ S390_IS_Z10 (stfle_bits), MEMPCPY_Z10)
+# endif
+# if HAVE_MEMCPY_Z900_G5
+ IFUNC_IMPL_ADD (array, i, mempcpy, 1, MEMPCPY_Z900_G5)
+# endif
+ )
+#endif /* HAVE_MEMCPY_IFUNC */

#ifdef HAVE_S390_VX_ASM_SUPPORT

diff --git a/sysdeps/s390/multiarch/memcpy-s390x.S b/sysdeps/s390/multiarch/memcpy-s390x.S
deleted file mode 100644
index b38caac72b..0000000000
--- a/sysdeps/s390/multiarch/memcpy-s390x.S
+++ /dev/null
@@ -1,132 +0,0 @@
-/* CPU specific memcpy implementations. 31/64 bit S/390 version.
- Copyright (C) 2012-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
-/* INPUT PARAMETERS
- %r2 = target operands address
- %r3 = source operands address
- %r4 = number of bytes to copy. */
-
- .text
-
-#if defined SHARED && IS_IN (libc)
-
-ENTRY(____mempcpy_z196)
- .machine "z196"
- .machinemode "zarch_nohighgprs"
- lgr %r1,%r2 # Use as dest
- la %r2,0(%r4,%r2) # Return dest + n
- j .L_Z196_start
-END(____mempcpy_z196)
-
-ENTRY(__memcpy_z196)
- .machine "z196"
- .machinemode "zarch_nohighgprs"
- lgr %r1,%r2 # r1: Use as dest ; r2: Return dest
-.L_Z196_start:
-# if !defined __s390x__
- llgfr %r4,%r4
-# endif /* !defined __s390x__ */
- ltgr %r4,%r4
- je .L_Z196_4
- aghi %r4,-1
- srlg %r5,%r4,8
- ltgr %r5,%r5
- jne .L_Z196_5
-.L_Z196_3:
- exrl %r4,.L_Z196_14
-.L_Z196_4:
- br %r14
-.L_Z196_5:
- cgfi %r5,262144 # Switch to mvcle for copies >64MB
- jh __memcpy_mvcle
-.L_Z196_2:
- pfd 1,768(%r3)
- pfd 2,768(%r1)
- mvc 0(256,%r1),0(%r3)
- aghi %r5,-1
- la %r1,256(%r1)
- la %r3,256(%r3)
- jne .L_Z196_2
- j .L_Z196_3
-.L_Z196_14:
- mvc 0(1,%r1),0(%r3)
-END(__memcpy_z196)
-
-ENTRY(____mempcpy_z10)
- .machine "z10"
- .machinemode "zarch_nohighgprs"
- lgr %r1,%r2 # Use as dest
- la %r2,0(%r4,%r2) # Return dest + n
- j .L_Z10_start
-END(____mempcpy_z10)
-
-ENTRY(__memcpy_z10)
- .machine "z10"
- .machinemode "zarch_nohighgprs"
- lgr %r1,%r2 # r1: Use as dest ; r2: Return dest
-.L_Z10_start:
-# if !defined __s390x__
- llgfr %r4,%r4
-# endif /* !defined __s390x__ */
- cgije %r4,0,.L_Z10_4
- aghi %r4,-1
- srlg %r5,%r4,8
- cgijlh %r5,0,.L_Z10_13
-.L_Z10_3:
- exrl %r4,.L_Z10_15
-.L_Z10_4:
- br %r14
-.L_Z10_13:
- cgfi %r5,65535 # Switch to mvcle for copies >16MB
- jh __memcpy_mvcle
-.L_Z10_12:
- pfd 1,768(%r3)
- pfd 2,768(%r1)
- mvc 0(256,%r1),0(%r3)
- la %r1,256(%r1)
- la %r3,256(%r3)
- brctg %r5,.L_Z10_12
- j .L_Z10_3
-.L_Z10_15:
- mvc 0(1,%r1),0(%r3)
-END(__memcpy_z10)
-
-# define __mempcpy ____mempcpy_default
-#endif /* SHARED && IS_IN (libc) */
-
-#define memcpy __memcpy_default
-#include "../memcpy.S"
-#undef memcpy
-
-#if defined SHARED && IS_IN (libc)
-.globl __GI_memcpy
-.set __GI_memcpy,__memcpy_default
-.globl __GI_mempcpy
-.set __GI_mempcpy,____mempcpy_default
-.globl __GI___mempcpy
-.set __GI___mempcpy,____mempcpy_default
-#else
-.globl memcpy
-.set memcpy,__memcpy_default
-.weak mempcpy
-.set mempcpy,__mempcpy
-#endif

[Stefan Liebler]

The ifunc handling for wmemset is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.
Glibc internal calls will use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wmemset variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wmemset variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wmemset.
* sysdeps/s390/multiarch/wmemset-c.c: Move to ...
* sysdeps/s390/wmemset-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wmemset-vx.S: Move to ...
* sysdeps/s390/wmemset-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wmemset.c: Move to ...
* sysdeps/s390/wmemset.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wmemset.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wmemset.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wmemset-c.c | 39 ++++++++++-------
sysdeps/s390/{multiarch => }/wmemset-vx.S | 22 +++++++---
sysdeps/s390/{multiarch => }/wmemset.c | 23 +++++++---
7 files changed, 126 insertions(+), 32 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wmemset.h
rename sysdeps/s390/{multiarch => }/wmemset-c.c (59%)
rename sysdeps/s390/{multiarch => }/wmemset-vx.S (91%)
rename sysdeps/s390/{multiarch => }/wmemset.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 17aabf328b..ae2ae3d2ea 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -99,5 +99,6 @@ sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsspn wcsspn-vx wcsspn-c \
wcspbrk wcspbrk-vx wcspbrk-c \
wcscspn wcscspn-vx wcscspn-c \
- wmemchr wmemchr-vx wmemchr-c
+ wmemchr wmemchr-vx wmemchr-c \
+ wmemset wmemset-vx wmemset-c
endif
diff --git a/sysdeps/s390/ifunc-wmemset.h b/sysdeps/s390/ifunc-wmemset.h
new file mode 100644
index 0000000000..c9d1d17c3b
--- /dev/null
+++ b/sysdeps/s390/ifunc-wmemset.h
@@ -0,0 +1,53 @@
+/* wmemset variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WMEMSET_IFUNC 1
+#else
+# define HAVE_WMEMSET_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WMEMSET_IFUNC_AND_VX_SUPPORT HAVE_WMEMSET_IFUNC
+#else
+# define HAVE_WMEMSET_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WMEMSET_DEFAULT WMEMSET_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WMEMSET_C 1
+# define HAVE_WMEMSET_Z13 1
+#else
+# define WMEMSET_DEFAULT WMEMSET_C
+# define HAVE_WMEMSET_C 1
+# define HAVE_WMEMSET_Z13 HAVE_WMEMSET_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WMEMSET_C
+# define WMEMSET_C __wmemset_c
+#else
+# define WMEMSET_C NULL
+#endif
+
+#if HAVE_WMEMSET_Z13
+# define WMEMSET_Z13 __wmemset_vx
+#else
+# define WMEMSET_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 92e28dc45d..cc6dd7adb1 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wmemset wmemset-vx wmemset-c \
- wmemcmp wmemcmp-vx wmemcmp-c
+sysdep_routines += wmemcmp wmemcmp-vx wmemcmp-c
endif

ifeq ($(subdir),iconvdata)
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index b5f55deb7f..7040959269 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -63,6 +63,7 @@
#include <ifunc-wcspbrk.h>
#include <ifunc-wcscspn.h>
#include <ifunc-wmemchr.h>
+#include <ifunc-wmemset.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -645,6 +646,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WMEMCHR_IFUNC */

+#if HAVE_WMEMSET_IFUNC
+ IFUNC_IMPL (i, name, wmemset,
+# if HAVE_WMEMSET_Z13
+ IFUNC_IMPL_ADD (array, i, wmemset,
+ dl_hwcap & HWCAP_S390_VX, WMEMSET_Z13)
+# endif
+# if HAVE_WMEMSET_C
+ IFUNC_IMPL_ADD (array, i, wmemset, 1, WMEMSET_C)
+# endif
+ )
+#endif /* HAVE_WMEMSET_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -653,8 +666,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wmemset);
-
IFUNC_VX_IMPL (wmemcmp);

#endif /* HAVE_S390_VX_ASM_SUPPORT */
diff --git a/sysdeps/s390/multiarch/wmemset-c.c b/sysdeps/s390/wmemset-c.c
similarity index 59%
rename from sysdeps/s390/multiarch/wmemset-c.c
rename to sysdeps/s390/wmemset-c.c
index 1969cf93dc..01e625496d 100644
--- a/sysdeps/s390/multiarch/wmemset-c.c
+++ b/sysdeps/s390/wmemset-c.c
@@ -16,22 +16,29 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WMEMSET __wmemset_c
-
-# include <wchar.h>
-extern __typeof (__wmemset) __wmemset_c;
-# undef weak_alias
-# define weak_alias(name, alias)
-# ifdef SHARED
-# undef libc_hidden_def
-# define libc_hidden_def(name) \
- __hidden_ver1 (__wmemset_c, __GI___wmemset, __wmemset_c);
-# undef libc_hidden_weak
-# define libc_hidden_weak(name) \
+#include <ifunc-wmemset.h>
+
+#if HAVE_WMEMSET_C
+# if HAVE_WMEMSET_IFUNC || HAVE_WMEMSET_Z13
+# define WMEMSET WMEMSET_C
+
+# undef weak_alias
+# define weak_alias(name, alias)
+
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_weak
+# define libc_hidden_weak(name)
+# undef libc_hidden_def
+# if ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define libc_hidden_def(name) \
+ __hidden_ver1 (__wmemset_c, __GI_wmemset, __wmemset_c) __attribute__((weak)); \
strong_alias (__wmemset_c, __wmemset_c_1); \
- __hidden_ver1 (__wmemset_c_1, __GI_wmemset, __wmemset_c_1);
-# endif /* SHARED */
+ __hidden_ver1 (__wmemset_c_1, __GI___wmemset, __wmemset_c_1);
+# else
+# define libc_hidden_def(name)
+# endif
+# endif
+# endif

# include <wcsmbs/wmemset.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/wmemset-vx.S b/sysdeps/s390/wmemset-vx.S
similarity index 91%
rename from sysdeps/s390/multiarch/wmemset-vx.S
rename to sysdeps/s390/wmemset-vx.S
index 0c2f6337b0..4b6050b5ac 100644
--- a/sysdeps/s390/multiarch/wmemset-vx.S
+++ b/sysdeps/s390/wmemset-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wmemset.h>
+#if HAVE_WMEMSET_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -38,7 +39,7 @@
-v17,v18,v19=copy of v16 for vstm
-v31=saved dest for return
*/
-ENTRY(__wmemset_vx)
+ENTRY(WMEMSET_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -137,6 +138,17 @@ ENTRY(__wmemset_vx)
br %r14
.Lfallback:
srlg %r4,%r4,2 /* Convert byte-count to character-count. */
- jg __wmemset_c
-END(__wmemset_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WMEMSET_C
+END(WMEMSET_Z13)
+
+# if ! HAVE_WMEMSET_IFUNC
+strong_alias (WMEMSET_Z13, __wmemset)
+weak_alias (__wmemset, wmemset)
+# endif
+
+# if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT \
+ && defined SHARED && IS_IN (libc)
+strong_alias (WMEMSET_Z13, __GI___wmemset)
+weak_alias (WMEMSET_Z13, __GI_wmemset)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wmemset.c b/sysdeps/s390/wmemset.c
similarity index 70%
rename from sysdeps/s390/multiarch/wmemset.c
rename to sysdeps/s390/wmemset.c
index 149b481470..6118754d1d 100644
--- a/sysdeps/s390/multiarch/wmemset.c
+++ b/sysdeps/s390/wmemset.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wmemset.h>
+
+#if HAVE_WMEMSET_IFUNC
# define wmemset __redirect_wmemset
# define __wmemset __redirect___wmemset
# include <wchar.h>
@@ -24,9 +26,18 @@
# undef __wmemset
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc_redirected (__redirect___wmemset, __wmemset)
-weak_alias (__wmemset, wmemset)
+# if HAVE_WMEMSET_C
+extern __typeof (__redirect___wmemset) WMEMSET_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wmemset.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WMEMSET_Z13
+extern __typeof (__redirect___wmemset) WMEMSET_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__redirect___wmemset, __wmemset,
+ (HAVE_WMEMSET_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WMEMSET_Z13
+ : WMEMSET_DEFAULT
+ )
+weak_alias (__wmemset, wmemset)
+#endif

[Stefan Liebler]

The ifunc handling for strcpy is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

Note: The fallback s390-32/s390-64 ifunc variants with mvst instruction
are now moved to the unified strcpy-z900.S file which can be used for
31/64bit. The s390-32/s390-64 files multiarch/strcpy.c and strcpy.S
are deleted.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strcpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strcpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strcpy.
* sysdeps/s390/multiarch/strcpy-vx.S: Move to ...
* sysdeps/s390/strcpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strcpy.c: Move to ...
* sysdeps/s390/strcpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strcpy.h: New file.
* sysdeps/s390/s390-64/strcpy.S: Move to ...
* sysdeps/s390/strcpy-z900.S: ... here and adjust to be usable
for 31/64bit and ifunc handling.
* sysdeps/s390/s390-32/multiarch/strcpy.c: Delete file.
* sysdeps/s390/s390-64/multiarch/strcpy.c: Likewise.
* sysdeps/s390/s390-32/strcpy.S: Likewise.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strcpy.h | 52 +++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 ++++-
sysdeps/s390/s390-32/multiarch/strcpy.c | 21 --------
sysdeps/s390/s390-32/strcpy.S | 36 -------------
sysdeps/s390/s390-64/multiarch/strcpy.c | 21 --------
sysdeps/s390/{multiarch => }/strcpy-vx.S | 26 +++++-----
.../s390/{s390-64/strcpy.S => strcpy-z900.S} | 33 +++++++++---
sysdeps/s390/{multiarch => }/strcpy.c | 17 +++++-
10 files changed, 123 insertions(+), 103 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strcpy.h
delete mode 100644 sysdeps/s390/s390-32/multiarch/strcpy.c
delete mode 100644 sysdeps/s390/s390-32/strcpy.S
delete mode 100644 sysdeps/s390/s390-64/multiarch/strcpy.c
rename sysdeps/s390/{multiarch => }/strcpy-vx.S (85%)
rename sysdeps/s390/{s390-64/strcpy.S => strcpy-z900.S} (66%)
rename sysdeps/s390/{multiarch => }/strcpy.c (69%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 4390af56b3..af9482adc3 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -61,5 +61,6 @@ sysdep_routines += bzero memset memset-z900 \
strstr strstr-vx strstr-c \
memmem memmem-vx memmem-c \
strlen strlen-vx strlen-c \
- strnlen strnlen-vx strnlen-c
+ strnlen strnlen-vx strnlen-c \
+ strcpy strcpy-vx strcpy-z900
endif
diff --git a/sysdeps/s390/ifunc-strcpy.h b/sysdeps/s390/ifunc-strcpy.h
new file mode 100644
index 0000000000..85e45556e6
--- /dev/null
+++ b/sysdeps/s390/ifunc-strcpy.h
@@ -0,0 +1,52 @@
+/* strcpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRCPY_IFUNC 1
+#else
+# define HAVE_STRCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRCPY_IFUNC_AND_VX_SUPPORT HAVE_STRCPY_IFUNC
+#else
+# define HAVE_STRCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRCPY_DEFAULT STRCPY_Z13
+# define HAVE_STRCPY_Z900_G5 0
+# define HAVE_STRCPY_Z13 1
+#else
+# define STRCPY_DEFAULT STRCPY_Z900_G5
+# define HAVE_STRCPY_Z900_G5 1
+# define HAVE_STRCPY_Z13 HAVE_STRCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRCPY_Z900_G5
+# define STRCPY_Z900_G5 __strcpy_default
+#else
+# define STRCPY_Z900_G5 NULL
+#endif
+
+#if HAVE_STRCPY_Z13
+# define STRCPY_Z13 __strcpy_vx
+#else
+# define STRCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 35ba223c5d..50f7f0b78d 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strcpy strcpy-vx \
- stpcpy stpcpy-vx stpcpy-c \
+sysdep_routines += stpcpy stpcpy-vx stpcpy-c \
strncpy strncpy-vx \
stpncpy stpncpy-vx stpncpy-c \
strcat strcat-vx strcat-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 680e5b738b..1784372db9 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -28,6 +28,7 @@
#include <ifunc-memmem.h>
#include <ifunc-strlen.h>
#include <ifunc-strnlen.h>
+#include <ifunc-strcpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -190,6 +191,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRNLEN_IFUNC */

+#if HAVE_STRCPY_IFUNC
+ IFUNC_IMPL (i, name, strcpy,
+# if HAVE_STRCPY_Z13
+ IFUNC_IMPL_ADD (array, i, strcpy,
+ dl_hwcap & HWCAP_S390_VX, STRCPY_Z13)
+# endif
+# if HAVE_STRCPY_Z900_G5
+ IFUNC_IMPL_ADD (array, i, strcpy, 1, STRCPY_Z900_G5)
+# endif
+ )
+#endif /* HAVE_STRCPY_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -202,7 +215,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcsnlen);

- IFUNC_VX_IMPL (strcpy);
IFUNC_VX_IMPL (wcscpy);

IFUNC_VX_IMPL (stpcpy);
diff --git a/sysdeps/s390/s390-32/multiarch/strcpy.c b/sysdeps/s390/s390-32/multiarch/strcpy.c
deleted file mode 100644
index 6a22e31a03..0000000000
--- a/sysdeps/s390/s390-32/multiarch/strcpy.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/* Multiple versions of strcpy.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* This wrapper-file is needed, because otherwise file
- sysdeps/s390/s390-[32|64]/strcpy.S will be used. */
-#include <sysdeps/s390/multiarch/strcpy.c>
diff --git a/sysdeps/s390/s390-32/strcpy.S b/sysdeps/s390/s390-32/strcpy.S
deleted file mode 100644
index d49136ee92..0000000000
--- a/sysdeps/s390/s390-32/strcpy.S
+++ /dev/null
@@ -1,36 +0,0 @@
-/* strcpy - copy a string from source to destination. For IBM S390
- This file is part of the GNU C Library.
- Copyright (C) 2000-2018 Free Software Foundation, Inc.
- Contributed by Martin Schwidefsky (***@de.ibm.com).
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/*
- * R2 = address of destination
- * R3 = address of source
- */
-
-#include "sysdep.h"
-#include "asm-syntax.h"
-
- .text
-ENTRY(strcpy)
- slr %r0,%r0
- lr %r1,%r2
-0: mvst %r1,%r3
- jo 0b
- br %r14
-END(strcpy)
-libc_hidden_builtin_def (strcpy)
diff --git a/sysdeps/s390/s390-64/multiarch/strcpy.c b/sysdeps/s390/s390-64/multiarch/strcpy.c
deleted file mode 100644
index 6a22e31a03..0000000000
--- a/sysdeps/s390/s390-64/multiarch/strcpy.c
+++ /dev/null
@@ -1,21 +0,0 @@
-/* Multiple versions of strcpy.
- Copyright (C) 2015-2018 Free Software Foundation, Inc.
- This file is part of the GNU C Library.
-
- The GNU C Library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2.1 of the License, or (at your option) any later version.
-
- The GNU C Library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with the GNU C Library; if not, see
- <http://www.gnu.org/licenses/>. */
-
-/* This wrapper-file is needed, because otherwise file
- sysdeps/s390/s390-[32|64]/strcpy.S will be used. */
-#include <sysdeps/s390/multiarch/strcpy.c>
diff --git a/sysdeps/s390/multiarch/strcpy-vx.S b/sysdeps/s390/strcpy-vx.S
similarity index 85%
rename from sysdeps/s390/multiarch/strcpy-vx.S
rename to sysdeps/s390/strcpy-vx.S
index 52197f57f7..844d23e4fe 100644
--- a/sysdeps/s390/multiarch/strcpy-vx.S
+++ b/sysdeps/s390/strcpy-vx.S
@@ -16,13 +16,13 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-
-# include "sysdep.h"
-# include "asm-syntax.h"
+#include <ifunc-strcpy.h>
+#include "sysdep.h"
+#include "asm-syntax.h"

.text

+#if HAVE_STRCPY_Z13
/* char * strcpy (const char *dest, const char *src)
Copy string src to dest.

@@ -36,7 +36,7 @@
-v17=index of zero
-v18=part of src
*/
-ENTRY(__strcpy_vx)
+ENTRY(STRCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -97,13 +97,13 @@ ENTRY(__strcpy_vx)
.Lfound_align:
vstl %v16,%r5,0(%r2) /* Copy characters including zero. */
br %r14
-END(__strcpy_vx)
+END(STRCPY_Z13)

-/* Use mvst-strcpy-implementation as default implementation. */
-# define strcpy __strcpy_c
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) strong_alias(__strcpy_c, __GI_strcpy)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+# if ! HAVE_STRCPY_IFUNC
+strong_alias (STRCPY_Z13, strcpy)
+# endif

-/* Include mvst-strcpy-implementation in s390-32/s390-64 subdirectory. */
-#include <strcpy.S>
+# if ! HAVE_STRCPY_Z900_G5 && defined SHARED && IS_IN (libc)
+strong_alias (STRCPY_Z13, __GI_strcpy)
+# endif
+#endif
diff --git a/sysdeps/s390/s390-64/strcpy.S b/sysdeps/s390/strcpy-z900.S
similarity index 66%
rename from sysdeps/s390/s390-64/strcpy.S
rename to sysdeps/s390/strcpy-z900.S
index 203c73c905..42798b1fd5 100644
--- a/sysdeps/s390/s390-64/strcpy.S
+++ b/sysdeps/s390/strcpy-z900.S
@@ -1,4 +1,4 @@
-/* strcpy - copy a string from source to destination. 64 bit S/390 version.
+/* strcpy - copy a string from source to destination. 64/31 bit S/390 version.
Copyright (C) 2001-2018 Free Software Foundation, Inc.
Contributed by Martin Schwidefsky (***@de.ibm.com).
This file is part of the GNU C Library.
@@ -21,15 +21,36 @@
%r2 = address of destination
%r3 = address of source. */

+#include <ifunc-strcpy.h>
#include "sysdep.h"
#include "asm-syntax.h"

+#if HAVE_STRCPY_Z900_G5
+# if defined __s390x__
+# define SLGR slgr
+# define LGR lgr
+# else
+# define SLGR slr
+# define LGR lr
+# endif /* ! defined __s390x__ */
+
.text
-ENTRY(strcpy)
- slgr %r0,%r0
- lgr %r1,%r2
+ENTRY(STRCPY_Z900_G5)
+ SLGR %r0,%r0
+ LGR %r1,%r2
0: mvst %r1,%r3
jo 0b
br %r14
-END(strcpy)
-libc_hidden_builtin_def (strcpy)
+END(STRCPY_Z900_G5)
+
+# undef SLGR
+# undef LGR
+
+# if ! HAVE_STRCPY_IFUNC
+strong_alias (STRCPY_Z900_G5, strcpy)
+# endif
+
+# if defined SHARED && IS_IN (libc)
+strong_alias (STRCPY_Z900_G5, __GI_strcpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/strcpy.c b/sysdeps/s390/strcpy.c
similarity index 69%
rename from sysdeps/s390/multiarch/strcpy.c
rename to sysdeps/s390/strcpy.c
index 8f32a13f67..f4e28e24c8 100644
--- a/sysdeps/s390/multiarch/strcpy.c
+++ b/sysdeps/s390/strcpy.c
@@ -16,12 +16,25 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strcpy.h>
+
+#if HAVE_STRCPY_IFUNC
# define strcpy __redirect_strcpy
# include <string.h>
# undef strcpy
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strcpy, __strcpy, strcpy)
+# if HAVE_STRCPY_Z900_G5
+extern __typeof (__redirect_strcpy) STRCPY_Z900_G5 attribute_hidden;
+# endif
+
+# if HAVE_STRCPY_Z13
+extern __typeof (__redirect_strcpy) STRCPY_Z13 attribute_hidden;
+# endif

+s390_libc_ifunc_expr (__redirect_strcpy, strcpy,
+ (HAVE_STRCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRCPY_Z13
+ : STRCPY_DEFAULT
+ )
#endif

[Stefan Liebler]

The ifunc handling for strspn is adjusted in order to omit ifunc
variants if those will never be used as the minimum architecture level
already supports newer CPUs by default.
Glibc internal calls will then also use the "newer" ifunc variant.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove strspn variants.
* sysdeps/s390/Makefile (sysdep_routines): Add strspn variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for strspn.
* sysdeps/s390/multiarch/strspn-c.c: Move to ...
* sysdeps/s390/strspn-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strspn-vx.S: Move to ...
* sysdeps/s390/strspn-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/strspn.c: Move to ...
* sysdeps/s390/strspn.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-strspn.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-strspn.h | 52 ++++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 14 ++++++-
sysdeps/s390/{multiarch => }/strspn-c.c | 18 ++++----
sysdeps/s390/{multiarch => }/strspn-vx.S | 19 +++++++--
sysdeps/s390/{multiarch => }/strspn.c | 21 +++++++---
7 files changed, 110 insertions(+), 20 deletions(-)
create mode 100644 sysdeps/s390/ifunc-strspn.h
rename sysdeps/s390/{multiarch => }/strspn-c.c (78%)
rename sysdeps/s390/{multiarch => }/strspn-vx.S (97%)
rename sysdeps/s390/{multiarch => }/strspn.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index c423c80a43..604ca68ef1 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -72,5 +72,6 @@ sysdep_routines += bzero memset memset-z900 \
strncmp strncmp-vx strncmp-c \
strchr strchr-vx strchr-c \
strchrnul strchrnul-vx strchrnul-c \
- strrchr strrchr-vx strrchr-c
+ strrchr strrchr-vx strrchr-c \
+ strspn strspn-vx strspn-c
endif
diff --git a/sysdeps/s390/ifunc-strspn.h b/sysdeps/s390/ifunc-strspn.h
new file mode 100644
index 0000000000..1152ba1f3d
--- /dev/null
+++ b/sysdeps/s390/ifunc-strspn.h
@@ -0,0 +1,52 @@
+/* strspn variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_STRSPN_IFUNC 1
+#else
+# define HAVE_STRSPN_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_STRSPN_IFUNC_AND_VX_SUPPORT HAVE_STRSPN_IFUNC
+#else
+# define HAVE_STRSPN_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define STRSPN_DEFAULT STRSPN_Z13
+# define HAVE_STRSPN_C 0
+# define HAVE_STRSPN_Z13 1
+#else
+# define STRSPN_DEFAULT STRSPN_C
+# define HAVE_STRSPN_C 1
+# define HAVE_STRSPN_Z13 HAVE_STRSPN_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_STRSPN_C
+# define STRSPN_C __strspn_c
+#else
+# define STRSPN_C NULL
+#endif
+
+#if HAVE_STRSPN_Z13
+# define STRSPN_Z13 __strspn_vx
+#else
+# define STRSPN_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index c826755558..9b141e338c 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),string)
-sysdep_routines += strspn strspn-vx strspn-c \
- strpbrk strpbrk-vx strpbrk-c \
+sysdep_routines += strpbrk strpbrk-vx strpbrk-c \
strcspn strcspn-vx strcspn-c \
memchr memchr-vx \
rawmemchr rawmemchr-vx rawmemchr-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 60cd705ffa..c39e1f793a 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -39,6 +39,7 @@
#include <ifunc-strchr.h>
#include <ifunc-strchrnul.h>
#include <ifunc-strrchr.h>
+#include <ifunc-strspn.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -333,6 +334,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_STRRCHR_IFUNC */

+#if HAVE_STRSPN_IFUNC
+ IFUNC_IMPL (i, name, strspn,
+# if HAVE_STRSPN_Z13
+ IFUNC_IMPL_ADD (array, i, strspn,
+ dl_hwcap & HWCAP_S390_VX, STRSPN_Z13)
+# endif
+# if HAVE_STRSPN_C
+ IFUNC_IMPL_ADD (array, i, strspn, 1, STRSPN_C)
+# endif
+ )
+#endif /* HAVE_STRSPN_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -367,7 +380,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,

IFUNC_VX_IMPL (wcsrchr);

- IFUNC_VX_IMPL (strspn);
IFUNC_VX_IMPL (wcsspn);

IFUNC_VX_IMPL (strpbrk);
diff --git a/sysdeps/s390/multiarch/strspn-c.c b/sysdeps/s390/strspn-c.c
similarity index 78%
rename from sysdeps/s390/multiarch/strspn-c.c
rename to sysdeps/s390/strspn-c.c
index 0efe61bfb2..506f668321 100644
--- a/sysdeps/s390/multiarch/strspn-c.c
+++ b/sysdeps/s390/strspn-c.c
@@ -16,13 +16,17 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define STRSPN __strspn_c
-# ifdef SHARED
-# undef libc_hidden_builtin_def
-# define libc_hidden_builtin_def(name) \
+#include <ifunc-strspn.h>
+
+#if HAVE_STRSPN_C
+# if HAVE_STRSPN_IFUNC
+# define STRSPN STRSPN_C
+# if defined SHARED && IS_IN (libc)
+# undef libc_hidden_builtin_def
+# define libc_hidden_builtin_def(name) \
__hidden_ver1 (__strspn_c, __GI_strspn, __strspn_c);
-# endif /* SHARED */
+# endif
+# endif

# include <string/strspn.c>
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+#endif
diff --git a/sysdeps/s390/multiarch/strspn-vx.S b/sysdeps/s390/strspn-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/strspn-vx.S
rename to sysdeps/s390/strspn-vx.S
index 6aa823e63b..ae5529b567 100644
--- a/sysdeps/s390/multiarch/strspn-vx.S
+++ b/sysdeps/s390/strspn-vx.S
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strspn.h>
+
+#if HAVE_STRSPN_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -57,7 +59,7 @@
otherwise =0;
r9: loaded byte count of vlbb accept-string
*/
-ENTRY(__strspn_vx)
+ENTRY(STRSPN_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -252,5 +254,14 @@ ENTRY(__strspn_vx)
Check for zero is in jump-target. */
j .Lslow_next_acc_notonbb /* ... and search for zero in
fully loaded vreg again. */
-END(__strspn_vx)
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+END(STRSPN_Z13)
+
+# if ! HAVE_STRSPN_IFUNC
+strong_alias (STRSPN_Z13, strspn)
+# endif
+
+# if ! HAVE_STRSPN_C && defined SHARED && IS_IN (libc)
+strong_alias (STRSPN_Z13, __GI_strspn)
+# endif
+
+#endif /* HAVE_STRSPN_Z13 */
diff --git a/sysdeps/s390/multiarch/strspn.c b/sysdeps/s390/strspn.c
similarity index 70%
rename from sysdeps/s390/multiarch/strspn.c
rename to sysdeps/s390/strspn.c
index bedbe98cfc..91401fdaf8 100644
--- a/sysdeps/s390/multiarch/strspn.c
+++ b/sysdeps/s390/strspn.c
@@ -16,7 +16,9 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-strspn.h>
+
+#if HAVE_STRSPN_IFUNC
# define strspn __redirect_strspn
/* Omit the strspn inline definitions because it would redefine strspn. */
# define __NO_STRING_INLINES
@@ -24,8 +26,17 @@
# undef strspn
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc2_redirected (__redirect_strspn, __strspn, strspn)
+# if HAVE_STRSPN_C
+extern __typeof (__redirect_strspn) STRSPN_C attribute_hidden;
+# endif
+
+# if HAVE_STRSPN_Z13
+extern __typeof (__redirect_strspn) STRSPN_Z13 attribute_hidden;
+# endif

-#else
-# include <string/strspn.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+s390_libc_ifunc_expr (__redirect_strspn, strspn,
+ (HAVE_STRSPN_Z13 && (hwcap & HWCAP_S390_VX))
+ ? STRSPN_Z13
+ : STRSPN_DEFAULT
+ )
+#endif /* HAVE_STRSPN_IFUNC */

[Stefan Liebler]

The s390/s390x memcpy implementations are safe to be
used by memmove. Starting with this commit, memmove is
using memcpy for the forward cases on s390.

ChangeLog:

* sysdeps/s390/memcopy.h: New file.
---
sysdeps/s390/memcopy.h | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
create mode 100644 sysdeps/s390/memcopy.h

diff --git a/sysdeps/s390/memcopy.h b/sysdeps/s390/memcopy.h
new file mode 100644
index 0000000000..9a76196502
--- /dev/null
+++ b/sysdeps/s390/memcopy.h
@@ -0,0 +1,23 @@
+/* memcopy.h -- definitions for memory copy functions.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <sysdeps/generic/memcopy.h>
+
+/* The s390/s390x memcpy implementations are safe to be used by memmove. */
+#undef MEMCPY_OK_FOR_FWD_MEMMOVE
+#define MEMCPY_OK_FOR_FWD_MEMMOVE 1

[Stefan Liebler]

Add a configure check for z13 in the same way as done for z196.

ChangeLog:

* config.h.in (HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT): New undefine.
* sysdeps/s390/configure.ac: Add check for z13 support.
* sysdeps/s390/configure: Regenerated.
---
config.h.in | 3 +++
sysdeps/s390/configure | 38 ++++++++++++++++++++++++++++++++++++++
sysdeps/s390/configure.ac | 28 ++++++++++++++++++++++++++++
3 files changed, 69 insertions(+)

diff --git a/config.h.in b/config.h.in
index beecc39d5b..422a6036ab 100644
--- a/config.h.in
+++ b/config.h.in
@@ -68,6 +68,9 @@
/* Define if assembler supports z196 zarch instructions as default on S390. */
#undef HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT

+/* Define if assembler supports z13 zarch instructions as default on S390. */
+#undef HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+
/* Define if assembler supports vector instructions on S390. */
#undef HAVE_S390_VX_ASM_SUPPORT

diff --git a/sysdeps/s390/configure b/sysdeps/s390/configure
index f30f864436..4a44775e30 100644
--- a/sysdeps/s390/configure
+++ b/sysdeps/s390/configure
@@ -187,5 +187,43 @@ then

fi

+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for S390 z13 zarch instruction support as default" >&5
+$as_echo_n "checking for S390 z13 zarch instruction support as default... " >&6; }
+if ${libc_cv_asm_s390_min_z13_zarch+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ cat > conftest.c <<\EOF
+int testinsn (void)
+{
+ int i;
+ __asm__ ("vl %%v16,0(%%r15)\n\t"
+ "vlgvf %0,%%v16,0"
+ : "=d" (i) : : "memory", "v16");
+ return i;
+}
+EOF
+if { ac_try='${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS --shared conftest.c
+ -o conftest.o &> /dev/null'
+ { { eval echo "\"\$as_me\":${as_lineno-$LINENO}: \"$ac_try\""; } >&5
+ (eval $ac_try) 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; } ;
+then
+ libc_cv_asm_s390_min_z13_zarch=yes
+else
+ libc_cv_asm_s390_min_z13_zarch=no
+fi
+rm -f conftest*
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $libc_cv_asm_s390_min_z13_zarch" >&5
+$as_echo "$libc_cv_asm_s390_min_z13_zarch" >&6; }
+
+if test "$libc_cv_asm_s390_min_z13_zarch" = yes ;
+then
+ $as_echo "#define HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT 1" >>confdefs.h
+
+fi
+
test -n "$critic_missing" && as_fn_error $? "
*** $critic_missing" "$LINENO" 5
diff --git a/sysdeps/s390/configure.ac b/sysdeps/s390/configure.ac
index 981f7a79dd..4dfb5574b4 100644
--- a/sysdeps/s390/configure.ac
+++ b/sysdeps/s390/configure.ac
@@ -135,5 +135,33 @@ then
AC_DEFINE(HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT)
fi

+AC_CACHE_CHECK(for S390 z13 zarch instruction support as default,
+ libc_cv_asm_s390_min_z13_zarch, [dnl
+cat > conftest.c <<\EOF
+int testinsn (void)
+{
+ int i;
+ __asm__ ("vl %%v16,0(%%r15)\n\t"
+ "vlgvf %0,%%v16,0"
+ : "=d" (i) : : "memory", "v16");
+ return i;
+}
+EOF
+dnl
+dnl test, if assembler supports S390 z13 zarch instructions as default
+if AC_TRY_COMMAND([${CC-cc} $CFLAGS $CPPFLAGS $LDFLAGS --shared conftest.c
+ -o conftest.o &> /dev/null]) ;
+then
+ libc_cv_asm_s390_min_z13_zarch=yes
+else
+ libc_cv_asm_s390_min_z13_zarch=no
+fi
+rm -f conftest* ])
+
+if test "$libc_cv_asm_s390_min_z13_zarch" = yes ;
+then
+ AC_DEFINE(HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT)
+fi
+
test -n "$critic_missing" && AC_MSG_ERROR([
*** $critic_missing])

[Stefan Liebler]

The ifunc handling for wcsncpy is adjusted in order to omit ifunc
if the minimum architecture level already supports newer CPUs by default.
Unfortunately the c ifunc variant can't be omitted at all as it is used
by the z13 ifunc variant as fallback if the pointers are not 4-byte aligned.

ChangeLog:

* sysdeps/s390/multiarch/Makefile
(sysdep_routines): Remove wcsncpy variants.
* sysdeps/s390/Makefile (sysdep_routines): Add wcsncpy variants.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Refactor ifunc handling for wcsncpy.
* sysdeps/s390/multiarch/wcsncpy-c.c: Move to ...
* sysdeps/s390/wcsncpy-c.c: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsncpy-vx.S: Move to ...
* sysdeps/s390/wcsncpy-vx.S: ... here and adjust ifunc handling.
* sysdeps/s390/multiarch/wcsncpy.c: Move to ...
* sysdeps/s390/wcsncpy.c: ... here and adjust ifunc handling.
* sysdeps/s390/ifunc-wcsncpy.h: New file.
---
sysdeps/s390/Makefile | 3 +-
sysdeps/s390/ifunc-wcsncpy.h | 53 +++++++++++++++++++++++
sysdeps/s390/multiarch/Makefile | 3 +-
sysdeps/s390/multiarch/ifunc-impl-list.c | 15 ++++++-
sysdeps/s390/{multiarch => }/wcsncpy-c.c | 10 +++--
sysdeps/s390/{multiarch => }/wcsncpy-vx.S | 17 +++++---
sysdeps/s390/{multiarch => }/wcsncpy.c | 23 +++++++---
7 files changed, 103 insertions(+), 21 deletions(-)
create mode 100644 sysdeps/s390/ifunc-wcsncpy.h
rename sysdeps/s390/{multiarch => }/wcsncpy-c.c (85%)
rename sysdeps/s390/{multiarch => }/wcsncpy-vx.S (97%)
rename sysdeps/s390/{multiarch => }/wcsncpy.c (70%)

diff --git a/sysdeps/s390/Makefile b/sysdeps/s390/Makefile
index 0fc0cdaee9..e10ad5b470 100644
--- a/sysdeps/s390/Makefile
+++ b/sysdeps/s390/Makefile
@@ -86,5 +86,6 @@ ifeq ($(subdir),wcsmbs)
sysdep_routines += wcslen wcslen-vx wcslen-c \
wcsnlen wcsnlen-vx wcsnlen-c \
wcscpy wcscpy-vx wcscpy-c \
- wcpcpy wcpcpy-vx wcpcpy-c
+ wcpcpy wcpcpy-vx wcpcpy-c \
+ wcsncpy wcsncpy-vx wcsncpy-c
endif
diff --git a/sysdeps/s390/ifunc-wcsncpy.h b/sysdeps/s390/ifunc-wcsncpy.h
new file mode 100644
index 0000000000..d7beca128a
--- /dev/null
+++ b/sysdeps/s390/ifunc-wcsncpy.h
@@ -0,0 +1,53 @@
+/* wcsncpy variant information on S/390 version.
+ Copyright (C) 2018 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#if defined USE_MULTIARCH && IS_IN (libc) \
+ && ! defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define HAVE_WCSNCPY_IFUNC 1
+#else
+# define HAVE_WCSNCPY_IFUNC 0
+#endif
+
+#ifdef HAVE_S390_VX_ASM_SUPPORT
+# define HAVE_WCSNCPY_IFUNC_AND_VX_SUPPORT HAVE_WCSNCPY_IFUNC
+#else
+# define HAVE_WCSNCPY_IFUNC_AND_VX_SUPPORT 0
+#endif
+
+#if defined HAVE_S390_MIN_Z13_ZARCH_ASM_SUPPORT
+# define WCSNCPY_DEFAULT WCSNCPY_Z13
+/* The z13 ifunc variant is using the common code variant as fallback! */
+# define HAVE_WCSNCPY_C 1
+# define HAVE_WCSNCPY_Z13 1
+#else
+# define WCSNCPY_DEFAULT WCSNCPY_C
+# define HAVE_WCSNCPY_C 1
+# define HAVE_WCSNCPY_Z13 HAVE_WCSNCPY_IFUNC_AND_VX_SUPPORT
+#endif
+
+#if HAVE_WCSNCPY_C
+# define WCSNCPY_C __wcsncpy_c
+#else
+# define WCSNCPY_C NULL
+#endif
+
+#if HAVE_WCSNCPY_Z13
+# define WCSNCPY_Z13 __wcsncpy_vx
+#else
+# define WCSNCPY_Z13 NULL
+#endif
diff --git a/sysdeps/s390/multiarch/Makefile b/sysdeps/s390/multiarch/Makefile
index 7d7b05dcf2..6631fd14d3 100644
--- a/sysdeps/s390/multiarch/Makefile
+++ b/sysdeps/s390/multiarch/Makefile
@@ -1,6 +1,5 @@
ifeq ($(subdir),wcsmbs)
-sysdep_routines += wcsncpy wcsncpy-vx wcsncpy-c \
- wcpncpy wcpncpy-vx wcpncpy-c \
+sysdep_routines += wcpncpy wcpncpy-vx wcpncpy-c \
wcscat wcscat-vx wcscat-c \
wcsncat wcsncat-vx wcsncat-c \
wcscmp wcscmp-vx wcscmp-c \
diff --git a/sysdeps/s390/multiarch/ifunc-impl-list.c b/sysdeps/s390/multiarch/ifunc-impl-list.c
index 656ab59db6..9ebaf4de6f 100644
--- a/sysdeps/s390/multiarch/ifunc-impl-list.c
+++ b/sysdeps/s390/multiarch/ifunc-impl-list.c
@@ -50,6 +50,7 @@
#include <ifunc-wcsnlen.h>
#include <ifunc-wcscpy.h>
#include <ifunc-wcpcpy.h>
+#include <ifunc-wcsncpy.h>

/* Maximum number of IFUNC implementations. */
#define MAX_IFUNC 3
@@ -476,6 +477,18 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
)
#endif /* HAVE_WCPCPY_IFUNC */

+#if HAVE_WCSNCPY_IFUNC
+ IFUNC_IMPL (i, name, wcsncpy,
+# if HAVE_WCSNCPY_Z13
+ IFUNC_IMPL_ADD (array, i, wcsncpy,
+ dl_hwcap & HWCAP_S390_VX, WCSNCPY_Z13)
+# endif
+# if HAVE_WCSNCPY_C
+ IFUNC_IMPL_ADD (array, i, wcsncpy, 1, WCSNCPY_C)
+# endif
+ )
+#endif /* HAVE_WCSNCPY_IFUNC */
+
#ifdef HAVE_S390_VX_ASM_SUPPORT

# define IFUNC_VX_IMPL(FUNC) \
@@ -484,8 +497,6 @@ __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array,
__##FUNC##_vx) \
IFUNC_IMPL_ADD (array, i, FUNC, 1, __##FUNC##_c))

- IFUNC_VX_IMPL (wcsncpy);
-
IFUNC_VX_IMPL (wcpncpy);

IFUNC_VX_IMPL (wcscat);
diff --git a/sysdeps/s390/multiarch/wcsncpy-c.c b/sysdeps/s390/wcsncpy-c.c
similarity index 85%
rename from sysdeps/s390/multiarch/wcsncpy-c.c
rename to sysdeps/s390/wcsncpy-c.c
index 6b89b8c14b..4d0ddb09ec 100644
--- a/sysdeps/s390/multiarch/wcsncpy-c.c
+++ b/sysdeps/s390/wcsncpy-c.c
@@ -16,10 +16,12 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
-# define WCSNCPY __wcsncpy_c
+#include <ifunc-wcsncpy.h>
+
+#if HAVE_WCSNCPY_C
+# if HAVE_WCSNCPY_IFUNC || HAVE_WCSNCPY_Z13
+# define WCSNCPY WCSNCPY_C
+# endif

-# include <wchar.h>
-extern __typeof (__wcsncpy) __wcsncpy_c;
# include <wcsmbs/wcsncpy.c>
#endif
diff --git a/sysdeps/s390/multiarch/wcsncpy-vx.S b/sysdeps/s390/wcsncpy-vx.S
similarity index 97%
rename from sysdeps/s390/multiarch/wcsncpy-vx.S
rename to sysdeps/s390/wcsncpy-vx.S
index b3400d50d9..9bcbdbf322 100644
--- a/sysdeps/s390/multiarch/wcsncpy-vx.S
+++ b/sysdeps/s390/wcsncpy-vx.S
@@ -16,7 +16,8 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsncpy.h>
+#if HAVE_WCSNCPY_Z13

# include "sysdep.h"
# include "asm-syntax.h"
@@ -40,7 +41,7 @@
-v18=part of src
-v31=register save area for r6, r7
*/
-ENTRY(__wcsncpy_vx)
+ENTRY(WCSNCPY_Z13)
.machine "z13"
.machinemode "zarch_nohighgprs"

@@ -217,7 +218,11 @@ ENTRY(__wcsncpy_vx)
j .Llt64

.Lfallback:
- jg __wcsncpy_c
-END(__wcsncpy_vx)
-
-#endif /* HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc) */
+ jg WCSNCPY_C
+END(WCSNCPY_Z13)
+
+# if ! HAVE_WCSNCPY_IFUNC
+strong_alias (WCSNCPY_Z13, __wcsncpy)
+weak_alias (__wcsncpy, wcsncpy)
+# endif
+#endif
diff --git a/sysdeps/s390/multiarch/wcsncpy.c b/sysdeps/s390/wcsncpy.c
similarity index 70%
rename from sysdeps/s390/multiarch/wcsncpy.c
rename to sysdeps/s390/wcsncpy.c
index 7209c7d431..e011de7ee7 100644
--- a/sysdeps/s390/multiarch/wcsncpy.c
+++ b/sysdeps/s390/wcsncpy.c
@@ -16,13 +16,24 @@
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */

-#if defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)
+#include <ifunc-wcsncpy.h>
+
+#if HAVE_WCSNCPY_IFUNC
# include <wchar.h>
# include <ifunc-resolve.h>

-s390_vx_libc_ifunc (__wcsncpy)
-weak_alias (__wcsncpy, wcsncpy)
+# if HAVE_WCSNCPY_C
+extern __typeof (__wcsncpy) WCSNCPY_C attribute_hidden;
+# endif

-#else
-# include <wcsmbs/wcsncpy.c>
-#endif /* !(defined HAVE_S390_VX_ASM_SUPPORT && IS_IN (libc)) */
+# if HAVE_WCSNCPY_Z13
+extern __typeof (__wcsncpy) WCSNCPY_Z13 attribute_hidden;
+# endif
+
+s390_libc_ifunc_expr (__wcsncpy, __wcsncpy,
+ (HAVE_WCSNCPY_Z13 && (hwcap & HWCAP_S390_VX))
+ ? WCSNCPY_Z13
+ : WCSNCPY_DEFAULT
+ )
+weak_alias (__wcsncpy, wcsncpy)
+#endif

[Stefan Liebler]

The ifunc macros s390_vx_libc* are no longer used and
can be removed as all users are now relying on
s390_libc_ifunc_expr.

The same applies to s390_libc_ifunc. The macro
s390_libc_ifunc_init is now renamed to
s390_libc_ifunc_expr_stfle_init and the users are
adjusted accordingly.

ChangeLog:

* sysdeps/s390/multiarch/ifunc-resolve.h
(s390_vx_libc_ifunc, s390_vx_libc_ifunc_redirected,
s390_vx_libc_ifunc2, s390_vx_libc_ifunc_init,
s390_vx_libc_ifunc2_redirected, s390_libc_ifunc):
Delete macro definition.
(s390_libc_ifunc_init): Rename to
s390_libc_ifunc_expr_stfle_init.
* sysdeps/s390/bzero: Use
s390_libc_ifunc_expr_stfle_init instead of
s390_libc_ifunc_init.
* sysdeps/s390/memcmp.c: Likewise.
* sysdeps/s390/memcpy.c: Likewise.
* sysdeps/s390/mempcpy.c: Likewise.
* sysdeps/s390/memset.c: Likewise.
---
sysdeps/s390/bzero.c | 2 +-
sysdeps/s390/memcmp.c | 2 +-
sysdeps/s390/memcpy.c | 2 +-
sysdeps/s390/mempcpy.c | 2 +-
sysdeps/s390/memset.c | 2 +-
sysdeps/s390/multiarch/ifunc-resolve.h | 37 +-------------------------
6 files changed, 6 insertions(+), 41 deletions(-)

diff --git a/sysdeps/s390/bzero.c b/sysdeps/s390/bzero.c
index 9f8d95781b..6b5d471c40 100644
--- a/sysdeps/s390/bzero.c
+++ b/sysdeps/s390/bzero.c
@@ -35,7 +35,7 @@ extern __typeof (__bzero) BZERO_Z196 attribute_hidden;

s390_libc_ifunc_expr (__bzero, __bzero,
({
- s390_libc_ifunc_init ();
+ s390_libc_ifunc_expr_stfle_init ();
(HAVE_MEMSET_Z196 && S390_IS_Z196 (stfle_bits))
? BZERO_Z196
: (HAVE_MEMSET_Z10 && S390_IS_Z10 (stfle_bits))
diff --git a/sysdeps/s390/memcmp.c b/sysdeps/s390/memcmp.c
index 952ff6af73..6d9276320a 100644
--- a/sysdeps/s390/memcmp.c
+++ b/sysdeps/s390/memcmp.c
@@ -37,7 +37,7 @@ extern __typeof (__redirect_memcmp) MEMCMP_Z196 attribute_hidden;

s390_libc_ifunc_expr (__redirect_memcmp, memcmp,
({
- s390_libc_ifunc_init ();
+ s390_libc_ifunc_expr_stfle_init ();
(HAVE_MEMCMP_Z196 && S390_IS_Z196 (stfle_bits))
? MEMCMP_Z196
: (HAVE_MEMCMP_Z10 && S390_IS_Z10 (stfle_bits))
diff --git a/sysdeps/s390/memcpy.c b/sysdeps/s390/memcpy.c
index 90a53ac27d..0ff24f18cf 100644
--- a/sysdeps/s390/memcpy.c
+++ b/sysdeps/s390/memcpy.c
@@ -38,7 +38,7 @@ extern __typeof (__redirect_memcpy) MEMCPY_Z196 attribute_hidden;

s390_libc_ifunc_expr (__redirect_memcpy, memcpy,
({
- s390_libc_ifunc_init ();
+ s390_libc_ifunc_expr_stfle_init ();
(HAVE_MEMCPY_Z196 && S390_IS_Z196 (stfle_bits))
? MEMCPY_Z196
: (HAVE_MEMCPY_Z10 && S390_IS_Z10 (stfle_bits))
diff --git a/sysdeps/s390/mempcpy.c b/sysdeps/s390/mempcpy.c
index a6a2373126..b687b33620 100644
--- a/sysdeps/s390/mempcpy.c
+++ b/sysdeps/s390/mempcpy.c
@@ -42,7 +42,7 @@ extern __typeof (__redirect___mempcpy) MEMPCPY_Z196 attribute_hidden;

s390_libc_ifunc_expr (__redirect___mempcpy, __mempcpy,
({
- s390_libc_ifunc_init ();
+ s390_libc_ifunc_expr_stfle_init ();
(HAVE_MEMCPY_Z196 && S390_IS_Z196 (stfle_bits))
? MEMPCPY_Z196
: (HAVE_MEMCPY_Z10 && S390_IS_Z10 (stfle_bits))
diff --git a/sysdeps/s390/memset.c b/sysdeps/s390/memset.c
index 57a35aebc7..75b011f1a2 100644
--- a/sysdeps/s390/memset.c
+++ b/sysdeps/s390/memset.c
@@ -37,7 +37,7 @@ extern __typeof (__redirect_memset) MEMSET_Z196 attribute_hidden;

s390_libc_ifunc_expr (__redirect_memset, memset,
({
- s390_libc_ifunc_init ();
+ s390_libc_ifunc_expr_stfle_init ();
(HAVE_MEMSET_Z196 && S390_IS_Z196 (stfle_bits))
? MEMSET_Z196
: (HAVE_MEMSET_Z10 && S390_IS_Z10 (stfle_bits))
diff --git a/sysdeps/s390/multiarch/ifunc-resolve.h b/sysdeps/s390/multiarch/ifunc-resolve.h
index 6139bd39c5..e4b71c66f6 100644
--- a/sysdeps/s390/multiarch/ifunc-resolve.h
+++ b/sysdeps/s390/multiarch/ifunc-resolve.h
@@ -40,7 +40,7 @@
".machine pop" "\n" \
: "=QS" (STFLE_BITS), "+d" (reg0) \
: : "cc");
-#define s390_libc_ifunc_init() \
+#define s390_libc_ifunc_expr_stfle_init() \
unsigned long long stfle_bits = 0ULL; \
if (__glibc_likely((hwcap & HWCAP_S390_STFLE) \
&& (hwcap & HWCAP_S390_ZARCH) \
@@ -49,41 +49,6 @@
S390_STORE_STFLE (stfle_bits); \
}

-#define s390_libc_ifunc(TYPE_FUNC, RESOLVERFUNC, FUNC) \
- /* Make the declarations of the optimized functions hidden in order
- to prevent GOT slots being generated for them. */ \
- extern __typeof (TYPE_FUNC) RESOLVERFUNC##_z196 attribute_hidden; \
- extern __typeof (TYPE_FUNC) RESOLVERFUNC##_z10 attribute_hidden; \
- extern __typeof (TYPE_FUNC) RESOLVERFUNC##_default attribute_hidden; \
- __ifunc (TYPE_FUNC, FUNC, \
- __glibc_likely (S390_IS_Z196 (stfle_bits)) \
- ? RESOLVERFUNC##_z196 \
- : __glibc_likely (S390_IS_Z10 (stfle_bits)) \
- ? RESOLVERFUNC##_z10 \
- : RESOLVERFUNC##_default, \
- unsigned long int hwcap, s390_libc_ifunc_init);
-
-#define s390_vx_libc_ifunc(FUNC) \
- s390_vx_libc_ifunc2_redirected(FUNC, FUNC, FUNC)
-
-#define s390_vx_libc_ifunc_redirected(TYPE_FUNC, FUNC) \
- s390_vx_libc_ifunc2_redirected(TYPE_FUNC, FUNC, FUNC)
-
-#define s390_vx_libc_ifunc2(RESOLVERFUNC, FUNC) \
- s390_vx_libc_ifunc2_redirected(FUNC, RESOLVERFUNC, FUNC)
-
-#define s390_vx_libc_ifunc_init()
-#define s390_vx_libc_ifunc2_redirected(TYPE_FUNC, RESOLVERFUNC, FUNC) \
- /* Make the declarations of the optimized functions hidden in order
- to prevent GOT slots being generated for them. */ \
- extern __typeof (TYPE_FUNC) RESOLVERFUNC##_vx attribute_hidden; \
- extern __typeof (TYPE_FUNC) RESOLVERFUNC##_c attribute_hidden; \
- __ifunc (TYPE_FUNC, FUNC, \
- (hwcap & HWCAP_S390_VX) \
- ? RESOLVERFUNC##_vx \
- : RESOLVERFUNC##_c, \
- unsigned long int hwcap, s390_vx_libc_ifunc_init);
-
#define s390_libc_ifunc_expr_init()
#define s390_libc_ifunc_expr(TYPE_FUNC, FUNC, EXPR) \
__ifunc (TYPE_FUNC, FUNC, EXPR, unsigned long int hwcap, \

[Adhemerval Zanella]

Post by Stefan Liebler
This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.
In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13". After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.
Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.
Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.
I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.
If no one objects, I plan to commit this series in the next one or two weeks.

The only issue I have for this change is it would require another
ABI variant for testing and validation, which would require more
coverage from build-many-glibcs.py (similar to armv7 for instance).

The gains I see is a slight reduction is loading time (due no ifunc)
and less code side. Is is what is driving you for this change? Does
it worth the extra testing and validation it might require?

[Stefan Liebler]

Post by Adhemerval Zanella

Post by Stefan Liebler
This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.
In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13". After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.
Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.
Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.
I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.
If no one objects, I plan to commit this series in the next one or two weeks.

The only issue I have for this change is it would require another
ABI variant for testing and validation, which would require more
coverage from build-many-glibcs.py (similar to armv7 for instance).

The ABI itself is not changed, but you are right tests could be done for
e.g. -march=zEC12 and -march=z13

Post by Adhemerval Zanella
The gains I see is a slight reduction is loading time (due no ifunc)
and less code side. Is is what is driving you for this change? Does
it worth the extra testing and validation it might require?

The current s390_vx_libc_ifunc macros are not flexible and do not allow
future enhancements as they only use the *_c or *_vx variant.

As soon as future ditros will use z13 as default architecture,
vector instructions can be used by default. But the current __GI_*
functions won't use the vector variants. Instead the fallback is always
used for internal calls and the vector variants are always used for
external calls via IFUNC.

Bye.
Stefan

[Adhemerval Zanella]

Post by Adhemerval Zanella

Post by Stefan Liebler
This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.
In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13".  After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.
Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.
Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.
I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.
If no one objects, I plan to commit this series in the next one or two weeks.

The only issue I have for this change is it would require another
ABI variant for testing and validation, which would require more
coverage from build-many-glibcs.py (similar to armv7 for instance).

The ABI itself is not changed, but you are right tests could be done for e.g. -march=zEC12 and -march=z13

I meant 'variant' in the sense it would require multiple glibc build to
actually validate a change in s390 implementations. If I am reading it
correctly, on memcpy, for instance, we might have:

1. HAVE_MEMCPY_IFUNC
1.1. HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
1.2. HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
1.3. Z900
2. !HAVE_MEMCPY_IFUNC
3. Disabled multi-arch

So is is basically 4 different glibc builds one will need to actually
test and check to fully validate s390.

Post by Adhemerval Zanella
The gains I see is a slight reduction is loading time (due no ifunc)
and less code side. Is is what is driving you for this change? Does
it worth the extra testing and validation it might require?

The current s390_vx_libc_ifunc macros are not flexible and do not allow future enhancements as they only use the *_c or *_vx variant.

Right, this is one issue which is not really tied to setting up different
build variants depending of the minimum ISA level.

As soon as future ditros will use z13 as default architecture,
vector instructions can be used by default.  But the current __GI_* functions won't use the vector variants.  Instead the fallback is always used for internal calls and the vector variants are always used for external calls via IFUNC.

Some ABI does impose restriction for ifunc with internal hidden symbols
(powerpc32 and i686 if I recall correctly), is it the case for s390
and/or s390x?

At least for some ABI, we can still route some internal symbols through
ifunc. On some x86 ifunc selector implementation the comments state it
might show no performance gain, but I am not sure about the validity of
the claims. So if s390 or s390x does not have any impeding reason, a
possibility might to use ifunc internally instead of relying on compiler
default or used options to setup the minimum ABI.

So the question is whether the possible internal symbols call optimization
worth that added complexity for build and validation.

[Stefan Liebler]

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Stefan Liebler
This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.
In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13".  After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.
Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.
Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.
I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.
If no one objects, I plan to commit this series in the next one or two weeks.

The only issue I have for this change is it would require another
ABI variant for testing and validation, which would require more
coverage from build-many-glibcs.py (similar to armv7 for instance).

The ABI itself is not changed, but you are right tests could be done for e.g. -march=zEC12 and -march=z13

I meant 'variant' in the sense it would require multiple glibc build to
actually validate a change in s390 implementations. If I am reading it
1. HAVE_MEMCPY_IFUNC
1.1. HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
1.2. HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
1.3. Z900
2. !HAVE_MEMCPY_IFUNC
3. Disabled multi-arch
So is is basically 4 different glibc builds one will need to actually
test and check to fully validate s390.

It depends on the change. If you change the implementation of the ifunc
variants and build with the oldest march level, then due to
ifunc-impl-list the tests will run all ifunc variants.

The patchset also do some clean up and unifies 31 vs 64bit
implementations from s390-32 and s390-64 folders. This simplifies
maintenance.

Post by Adhemerval Zanella

Post by Adhemerval Zanella
The gains I see is a slight reduction is loading time (due no ifunc)
and less code side. Is is what is driving you for this change? Does
it worth the extra testing and validation it might require?

The current s390_vx_libc_ifunc macros are not flexible and do not allow future enhancements as they only use the *_c or *_vx variant.

Right, this is one issue which is not really tied to setting up different
build variants depending of the minimum ISA level.

As soon as future ditros will use z13 as default architecture,
vector instructions can be used by default.  But the current __GI_* functions won't use the vector variants.  Instead the fallback is always used for internal calls and the vector variants are always used for external calls via IFUNC.

Some ABI does impose restriction for ifunc with internal hidden symbols
(powerpc32 and i686 if I recall correctly), is it the case for s390
and/or s390x?

On s390 the compiler will generate e.g. a PC32DBL relocation instead of
PLT32DBL one and the GOT pointer might not be setup in r12. But the
called PLT stub relies on a setup r12.

And on older binutils (unfortunately I'm not sure if it was version >=
or < 2.25) bugs regarding ifunc lead to a segmentation fault while
linking libc.so.

Post by Adhemerval Zanella
At least for some ABI, we can still route some internal symbols through
ifunc. On some x86 ifunc selector implementation the comments state it
might show no performance gain, but I am not sure about the validity of
the claims. So if s390 or s390x does not have any impeding reason, a
possibility might to use ifunc internally instead of relying on compiler
default or used options to setup the minimum ABI.

If I start routing those symbols via IFUNC then an additional PLT stub
is introduced in libc.so. If we are running on >=z13 then the __XYZ_vx
is called via PLT. On machines <z13 __XYZ_c is also called via PLT.
Compared to non-ifunc __GI_ symbols we have introduced the extra PLT
stub overhead without any advantage on <z13 machines as the __XYZ_c
implementation is called.

Post by Adhemerval Zanella
So the question is whether the possible internal symbols call optimization
worth that added complexity for build and validation.

E.g. for strstr there is an advantage. On s390 we are jumping from the
common-code implementations to the vector implementations. On other
archs we would jump from one vector implementation to another vector
implementation and you perhaps don't see so much difference.

[Adhemerval Zanella]

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Stefan Liebler
This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.
In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13".  After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.
Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.
Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.
I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.
If no one objects, I plan to commit this series in the next one or two weeks.

The only issue I have for this change is it would require another
ABI variant for testing and validation, which would require more
coverage from build-many-glibcs.py (similar to armv7 for instance).

The ABI itself is not changed, but you are right tests could be done for e.g. -march=zEC12 and -march=z13

I meant 'variant' in the sense it would require multiple glibc build to
actually validate a change in s390 implementations.  If I am reading it
   1. HAVE_MEMCPY_IFUNC
     1.1. HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
     1.2. HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
     1.3. Z900
   2. !HAVE_MEMCPY_IFUNC
   3. Disabled multi-arch
So is is basically 4 different glibc builds one will need to actually
test and check to fully validate s390.

It depends on the change. If you change the implementation of the ifunc variants and build with the oldest march level, then due to ifunc-impl-list the tests will run all ifunc variants.

You validate the ifunc implementation itself, you need to also check if it
builds on all the variants (even when you are sure that the ifunc variant
itself works). That's what I am trying to avoid, to have multiple possible
glibc build depending of how you configure the compiler.

The patchset also do some clean up and unifies 31 vs 64bit implementations from s390-32 and s390-64 folders. This simplifies maintenance.

I do agree with these kind of changes.

Post by Adhemerval Zanella

Post by Adhemerval Zanella
The gains I see is a slight reduction is loading time (due no ifunc)
and less code side. Is is what is driving you for this change? Does
it worth the extra testing and validation it might require?

The current s390_vx_libc_ifunc macros are not flexible and do not allow future enhancements as they only use the *_c or *_vx variant.

Right, this is one issue which is not really tied to setting up different
build variants depending of the minimum ISA level.

As soon as future ditros will use z13 as default architecture,
vector instructions can be used by default.  But the current __GI_* functions won't use the vector variants.  Instead the fallback is always used for internal calls and the vector variants are always used for external calls via IFUNC.

Some ABI does impose restriction for ifunc with internal hidden symbols
(powerpc32 and i686 if I recall correctly), is it the case for s390
and/or s390x?

On s390 the compiler will generate e.g. a PC32DBL relocation instead of PLT32DBL one and the GOT pointer might not be setup in r12. But the called PLT stub relies on a setup r12.
And on older binutils (unfortunately I'm not sure if it was version >= or < 2.25) bugs regarding ifunc lead to a segmentation fault while linking libc.so.

Do you consider this a deal-break issue? One option is to check the minimum
supported binutils for this change and set it as the minimum required one,
another is disable internal ifunc for such ABI (as for powerpc32, powerpc64
does work).

Post by Adhemerval Zanella
At least for some ABI, we can still route some internal symbols through
ifunc. On some x86 ifunc selector implementation the comments state it
might show no performance gain, but I am not sure about the validity of
the claims. So if s390 or s390x does not have any impeding reason, a
possibility might to use ifunc internally instead of relying on compiler
default or used options to setup the minimum ABI.

If I start routing those symbols via IFUNC then an additional PLT stub is introduced in libc.so. If we are running on >=z13 then the __XYZ_vx is called via PLT. On machines <z13 __XYZ_c is also called via PLT. Compared to non-ifunc __GI_ symbols we have introduced the extra PLT stub overhead without any advantage on <z13 machines as the __XYZ_c implementation is called.

Do we have real usercases where they are stressing libc symbols that are
doing intra-symbol calls and which PLT overhead is dominant?

Post by Adhemerval Zanella
So the question is whether the possible internal symbols call optimization
worth that added complexity for build and validation.

E.g. for strstr there is an advantage. On s390 we are jumping from the common-code implementations to the vector implementations. On other archs we would jump from one vector implementation to another vector implementation and you perhaps don't see so much difference.

One option maybe is maybe to trade some code size for a specific strstr
variant for z13, which calls strstr/strnlen/memcpy locally. Specially
for strstr, is the difference really worth to add this build and validation
complexity?

[Stefan Liebler]

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Stefan Liebler
This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.
In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13".  After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.
Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.
Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.
I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.
If no one objects, I plan to commit this series in the next one or two weeks.

The only issue I have for this change is it would require another
ABI variant for testing and validation, which would require more
coverage from build-many-glibcs.py (similar to armv7 for instance).

The ABI itself is not changed, but you are right tests could be done for e.g. -march=zEC12 and -march=z13

I meant 'variant' in the sense it would require multiple glibc build to
actually validate a change in s390 implementations.  If I am reading it
   1. HAVE_MEMCPY_IFUNC
     1.1. HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
     1.2. HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
     1.3. Z900
   2. !HAVE_MEMCPY_IFUNC
   3. Disabled multi-arch
So is is basically 4 different glibc builds one will need to actually
test and check to fully validate s390.

It depends on the change. If you change the implementation of the ifunc variants and build with the oldest march level, then due to ifunc-impl-list the tests will run all ifunc variants.

You validate the ifunc implementation itself, you need to also check if it
builds on all the variants (even when you are sure that the ifunc variant
itself works). That's what I am trying to avoid, to have multiple possible
glibc build depending of how you configure the compiler.

You always have to build and test glibc with your used
compiler-configuration of the target distro. E.g. using z13 as default
architecture level set can also introduce other bugs which are not
triggered if the test was previously done with zEC12 as als.

A caller of a function is just calling __GI_XYZ. If you change the
caller side then you rely on the correct __GI_XYZ symbol.
If you change one existing implementation of XYZ, then the __GI_
mechanism is not touched and you have to test the changed implementation.
If you add a new implementation for a new machine, then you have to
extend and test the mechanism. But then you can also afford to build
multiple glibcs. If __GI_XYZ exists twice or does not exist, then you'll
get a link error without running the testsuite. If __GI_XYZ points to a
false implementation, this is really bad. Then you'll either see it with
readelf or it fails as you are running on an too old machine. But a
change in the ifunc-resolver can also lead to such an issue.
Are all the power[4 to X], multiarch and co variants tested on every change?

Post by Adhemerval Zanella

The patchset also do some clean up and unifies 31 vs 64bit implementations from s390-32 and s390-64 folders. This simplifies maintenance.

I do agree with these kind of changes.

Post by Adhemerval Zanella

Post by Adhemerval Zanella
The gains I see is a slight reduction is loading time (due no ifunc)
and less code side. Is is what is driving you for this change? Does
it worth the extra testing and validation it might require?

The current s390_vx_libc_ifunc macros are not flexible and do not allow future enhancements as they only use the *_c or *_vx variant.

Right, this is one issue which is not really tied to setting up different
build variants depending of the minimum ISA level.

As soon as future ditros will use z13 as default architecture,
vector instructions can be used by default.  But the current __GI_* functions won't use the vector variants.  Instead the fallback is always used for internal calls and the vector variants are always used for external calls via IFUNC.

Some ABI does impose restriction for ifunc with internal hidden symbols
(powerpc32 and i686 if I recall correctly), is it the case for s390
and/or s390x?

On s390 the compiler will generate e.g. a PC32DBL relocation instead of PLT32DBL one and the GOT pointer might not be setup in r12. But the called PLT stub relies on a setup r12.
And on older binutils (unfortunately I'm not sure if it was version >= or < 2.25) bugs regarding ifunc lead to a segmentation fault while linking libc.so.

Do you consider this a deal-break issue? One option is to check the minimum
supported binutils for this change and set it as the minimum required one,
another is disable internal ifunc for such ABI (as for powerpc32, powerpc64
does work).

The "not setuped r12 register" is no binutils-bug. This is such an
restriction with ifunc and internal hidden symbols you've mentioned.
Thus I can't use IFUNC __GI_* symbols.

I disable internal ifunc by setting the __GI_* symbols to the "oldest"
implementation. Or is there another mechanism?

Post by Adhemerval Zanella

Post by Adhemerval Zanella
At least for some ABI, we can still route some internal symbols through
ifunc. On some x86 ifunc selector implementation the comments state it
might show no performance gain, but I am not sure about the validity of
the claims. So if s390 or s390x does not have any impeding reason, a
possibility might to use ifunc internally instead of relying on compiler
default or used options to setup the minimum ABI.

If I start routing those symbols via IFUNC then an additional PLT stub is introduced in libc.so. If we are running on >=z13 then the __XYZ_vx is called via PLT. On machines <z13 __XYZ_c is also called via PLT. Compared to non-ifunc __GI_ symbols we have introduced the extra PLT stub overhead without any advantage on <z13 machines as the __XYZ_c implementation is called.

Do we have real usercases where they are stressing libc symbols that are
doing intra-symbol calls and which PLT overhead is dominant?

Usually we try to avoid PLT calls. I think this is one idea behind the
__GI_* symbols. Shall we remove those symbols and always jump over PLT?

Post by Adhemerval Zanella

Post by Adhemerval Zanella
So the question is whether the possible internal symbols call optimization
worth that added complexity for build and validation.

E.g. for strstr there is an advantage. On s390 we are jumping from the common-code implementations to the vector implementations. On other archs we would jump from one vector implementation to another vector implementation and you perhaps don't see so much difference.

One option maybe is maybe to trade some code size for a specific strstr
variant for z13, which calls strstr/strnlen/memcpy locally. Specially
for strstr, is the difference really worth to add this build and validation
complexity?

[Adhemerval Zanella]

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Stefan Liebler
This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.
In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13".  After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.
Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.
Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.
I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.
If no one objects, I plan to commit this series in the next one or two weeks.

The only issue I have for this change is it would require another
ABI variant for testing and validation, which would require more
coverage from build-many-glibcs.py (similar to armv7 for instance).

The ABI itself is not changed, but you are right tests could be done for e.g. -march=zEC12 and -march=z13

I meant 'variant' in the sense it would require multiple glibc build to
actually validate a change in s390 implementations.  If I am reading it
    1. HAVE_MEMCPY_IFUNC
      1.1. HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
      1.2. HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
      1.3. Z900
    2. !HAVE_MEMCPY_IFUNC
    3. Disabled multi-arch
So is is basically 4 different glibc builds one will need to actually
test and check to fully validate s390.

It depends on the change. If you change the implementation of the ifunc variants and build with the oldest march level, then due to ifunc-impl-list the tests will run all ifunc variants.

You validate the ifunc implementation itself, you need to also check if it
builds on all the variants (even when you are sure that the ifunc variant
itself works). That's what I am trying to avoid, to have multiple possible
glibc build depending of how you configure the compiler.

You always have to build and test glibc with your used compiler-configuration of the target distro. E.g. using z13 as default architecture level set can also introduce other bugs which are not triggered if the test was previously done with zEC12 as als.
A caller of a function is just calling __GI_XYZ. If you change the caller side then you rely on the correct __GI_XYZ symbol.
If you change one existing implementation of XYZ, then the __GI_ mechanism is not touched and you have to test the changed implementation.
If you add a new implementation for a new machine, then you have to extend and test the mechanism. But then you can also afford to build multiple glibcs. If __GI_XYZ exists twice or does not exist, then you'll get a link error without running the testsuite. If __GI_XYZ points to a false implementation, this is really bad. Then you'll either see it with readelf or it fails as you are running on an too old machine. But a change in the ifunc-resolver can also lead to such an issue.

I am not really following your rationale here, the idea is to trade off
between add build complexity to squeeze some performance versus have a
more simple and predictable configure builds.

My point is I would like to have some measurements if this micro-optimization
really pays for the effort of adding even more build and validation
complexity.

Are all the power[4 to X], multiarch and co variants tested on every change?

I would expected so, however I also expect that most distribution just use
default build option (which use ifunc).

The powerpc situation is not really optimal imho and it is an artefact from
how distro usually deployed glibc for different powerpc chips (multiple
builds with loader selecting the path depending of the hwcap value). I really
would like we we move to deprecate and remove the --with-cpu option, so we
have basically just two variants (enable and disable multiarch).

The power4-only multiarch support was motivated mainly because it is the
minimum chip with support for hp-timing. I am not sure it was the best option
back then, but it indeed simplified how to internally handle the minimum ABI
requirements. One option might indeed use a similar strategy to enable disable
hp-timing.h based on compiler support.

Now for powerpc64le I do see that this kind of refactor make sense, since it
uses the old powerpc64 internal organization. Since the minimum ISA requirement
for powerpc64le is 2.07, it does not make sense pack all old ISA optimization
for ifunc variants.

Post by Adhemerval Zanella

The patchset also do some clean up and unifies 31 vs 64bit implementations from s390-32 and s390-64 folders. This simplifies maintenance.

I do agree with these kind of changes.

Post by Adhemerval Zanella

Post by Adhemerval Zanella
The gains I see is a slight reduction is loading time (due no ifunc)
and less code side. Is is what is driving you for this change? Does
it worth the extra testing and validation it might require?

The current s390_vx_libc_ifunc macros are not flexible and do not allow future enhancements as they only use the *_c or *_vx variant.

Right, this is one issue which is not really tied to setting up different
build variants depending of the minimum ISA level.

As soon as future ditros will use z13 as default architecture,
vector instructions can be used by default.  But the current __GI_* functions won't use the vector variants.  Instead the fallback is always used for internal calls and the vector variants are always used for external calls via IFUNC.

Some ABI does impose restriction for ifunc with internal hidden symbols
(powerpc32 and i686 if I recall correctly), is it the case for s390
and/or s390x?

On s390 the compiler will generate e.g. a PC32DBL relocation instead of PLT32DBL one and the GOT pointer might not be setup in r12. But the called PLT stub relies on a setup r12.
And on older binutils (unfortunately I'm not sure if it was version >= or < 2.25) bugs regarding ifunc lead to a segmentation fault while linking libc.so.

Do you consider this a deal-break issue? One option is to check the minimum
supported binutils for this change and set it as the minimum required one,
another is disable internal ifunc for such ABI (as for powerpc32, powerpc64
does work).

The "not setuped r12 register" is no binutils-bug. This is such an restriction with ifunc and internal hidden symbols you've mentioned.
Thus I can't use IFUNC __GI_* symbols.
I disable internal ifunc by setting the __GI_* symbols to the "oldest" implementation. Or is there another mechanism?

Is is an issue for both s390 and s390x?

Post by Adhemerval Zanella

Post by Adhemerval Zanella
At least for some ABI, we can still route some internal symbols through
ifunc. On some x86 ifunc selector implementation the comments state it
might show no performance gain, but I am not sure about the validity of
the claims. So if s390 or s390x does not have any impeding reason, a
possibility might to use ifunc internally instead of relying on compiler
default or used options to setup the minimum ABI.

If I start routing those symbols via IFUNC then an additional PLT stub is introduced in libc.so. If we are running on >=z13 then the __XYZ_vx is called via PLT. On machines <z13 __XYZ_c is also called via PLT. Compared to non-ifunc __GI_ symbols we have introduced the extra PLT stub overhead without any advantage on <z13 machines as the __XYZ_c implementation is called.

Do we have real usercases where they are stressing libc symbols that are
doing intra-symbol calls and which PLT overhead is dominant?

Usually we try to avoid PLT calls. I think this is one idea behind the __GI_* symbols. Shall we remove those symbols and always jump over PLT?

It really depends of the implementation that is calling and if it might be
a hotspot. Again I really want to understand if this change is just for the
sake of a micro-optimization or if you really targetting a real usercase
where internal __GI_ symbols are being a hotspot.

In any case, what do you think about reworking the patch to refactor s390
ifunc to be able to handle more variants, unify the code, and add z13
variants and start a separate thread about how to handle the ifunc and
compiler default options? I really want to check with distro focal points
and other maintainers if my objections does indeed does make sense.

Post by Adhemerval Zanella

Post by Adhemerval Zanella
So the question is whether the possible internal symbols call optimization
worth that added complexity for build and validation.

E.g. for strstr there is an advantage. On s390 we are jumping from the common-code implementations to the vector implementations. On other archs we would jump from one vector implementation to another vector implementation and you perhaps don't see so much difference.

One option maybe is maybe to trade some code size for a specific strstr
variant for z13, which calls strstr/strnlen/memcpy locally.  Specially
for strstr, is the difference really worth to add this build and validation
complexity?

[Joseph Myers]

Post by Adhemerval Zanella
builds with loader selecting the path depending of the hwcap value). I really
would like we we move to deprecate and remove the --with-cpu option, so we
have basically just two variants (enable and disable multiarch).

I'd like to deprecate and remove --with-cpu - but replace it with the
default CPU being automatically determined by how the compiler behaves
(see <https://sourceware.org/ml/libc-alpha/2017-02/msg00154.html>).

Avoiding building in lots of function variants for CPUs older than the
minimum supported still seems appropriate to me. What I don't know is if
there is a good way to design things with the following properties:

* IFUNC function variants are handled the same way for all architectures.

* A function is automatically not an IFUNC if the minimum supported CPU
implies one particular version would always be used.

* If more than one function might be used based on the minimum supported
CPU, all those that wouldn't ever be used automatically get compiled out.

* A single optimal version (choice based on the compiler's code
generation) is automatically selected in the non-multiarch case.

[Stefan Liebler]

Post by Joseph Myers

Post by Adhemerval Zanella
builds with loader selecting the path depending of the hwcap value). I really
would like we we move to deprecate and remove the --with-cpu option, so we
have basically just two variants (enable and disable multiarch).

I'd like to deprecate and remove --with-cpu - but replace it with the
default CPU being automatically determined by how the compiler behaves
(see <https://sourceware.org/ml/libc-alpha/2017-02/msg00154.html>).

The current approach on s390 is not to use submachine but to have
configure checks for each needed CPU which are responsible for setting
macro defines in config.h. Those checks are e.g. using an inline
assembly with a vector instruction to determine if z13 vector
instructions are supported as default. E.g. gcc -march=z13 -msoft-float
would lead to frecord-gcc-switches-output:
.ascii "-march=z13"
...
.ascii "-msoft-float"
But the assembler fails with "Error: Unrecognized opcode
<vector-instruction>" as the vector instructions are implicitly disabled
by -msoft-float.

Post by Joseph Myers
Avoiding building in lots of function variants for CPUs older than the
minimum supported still seems appropriate to me. What I don't know is if
* IFUNC function variants are handled the same way for all architectures.

I assume all are using the __ifunc macros from include/libc-symbols.h.

Some archs are using an ifunc'ed __GI_* symbol which jumps over
plt-stub. Some archs like s390-32 can't do that as the plt-stub needs a
setuped GOT pointer in r12. But r12 does not need to be setuped for
usual internal calls. Thus the __GI_* symbol is equivalent to the oldest
available ifunc variant.

Unfortunately defining the __GI_* symbol needs to be done in the compile
unit of the used ifunc variant or the ifunc'ed symbol itself. Therefore
this logic is spread over multiple places instead of one common place
(perhaps within the __ifunc macro).

Post by Joseph Myers
* A function is automatically not an IFUNC if the minimum supported CPU
implies one particular version would always be used.
* If more than one function might be used based on the minimum supported
CPU, all those that wouldn't ever be used automatically get compiled out.
* A single optimal version (choice based on the compiler's code
generation) is automatically selected in the non-multiarch case.

My patches are introducing ifunc-*.h files which decides if ifunc is
needed for a specific function depending on no-/multiarch, libc/ldso and
the minimum supported CPU. At the end there is one macro
HAVE_<FUNCTION>_IFUNC with the value 0 or 1.

Afterwards there is a decision which ifunc variants will be available
depending on the minimum supported CPU and if we have ifunc or not.
Older variants than the minimum supported CPU won't be available. At the
end there are multiple macros HAVE_<FUNCTION>_<VARIANT> with value 0 or
1. E.g. if multiarch is not available there is only one macro per
function with 1 and all others are 0.

The ifunc-*.h files are included in several files and those macros are
used to:
-compile or not compile a specific variant
-generating ifunc-resolver expressions which only choose between the
available ifunc variants and generating the ifunc-symbol or just an
alias to the one available variant.
-using the minimum available ifunc variant as the "__GI_* symbol".
-if ifunc is needed, the available ifunc variants are added in
ifunc-impl-list.

As it depends on the used gcc/binutils which parts (variants,
ifunc-resolver, ...) will be needed, either all the stuff needs to be in
one file like sysdeps/arch/function.x or there will be multiple files
specified as sysdep_routines in the Makefile.

As you can't mix a function implementation (in an assembler file) and
the __ifunc macro (c-code) you'll automatically need multiple files.

The multiarch subdirectory cannot be used to distinguish "default"- vs
"enhanced"-ifunc-variants. There is not the one "default"-variant as
this depends on the used gcc/binutils. Thus all files are moved to
sysdeps/s390/ and will always be build. But depending on the needed
parts some resulting object-files can be empty.

[Stefan Liebler]

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Stefan Liebler
This patch series is mainly refactoring the s390 specific ifunc handling.
The idea is to omit ifunc variants or ifunc at all if the used compile
options are already building for newer cpus. The glibc internal calls
and ld.so will then use "newer" ifunc variants as before.
In case of the memcpy, memset and memcmp functions, the newest ifunc variant
is for z196 and there are two further ones for older cpus, but the current usual
compile options are e.g. building for zEC12.
In case of the string / wcsmbs functions, there are variants for z13 and
"before z13".  After switching to z13 as default cpu level, there won't
be IFUNC symbols in s390 libc.so.
Furthermore new z13 specific ifunc variants are introduced for
memmove, strstr and memmem.
Some functions like the mem* functions are duplicated twice for 31 and 64 bit.
In fact they are nearly the same. Thus those implementations are now unified
and adjusted in order to be usable for 31 and 64bit.
I've build and tested these patches with different -march levels
and with / without multiarch and checked the symbols with readelf - e.g. if
IFUNC is used or not and if the __GI_ symbols are targeting the correct
ifunc variant.
If no one objects, I plan to commit this series in the next one or two weeks.

The only issue I have for this change is it would require another
ABI variant for testing and validation, which would require more
coverage from build-many-glibcs.py (similar to armv7 for instance).

The ABI itself is not changed, but you are right tests could be done for e.g. -march=zEC12 and -march=z13

I meant 'variant' in the sense it would require multiple glibc build to
actually validate a change in s390 implementations.  If I am reading it
    1. HAVE_MEMCPY_IFUNC
      1.1. HAVE_S390_MIN_Z196_ZARCH_ASM_SUPPORT
      1.2. HAVE_S390_MIN_Z10_ZARCH_ASM_SUPPORT
      1.3. Z900
    2. !HAVE_MEMCPY_IFUNC
    3. Disabled multi-arch
So is is basically 4 different glibc builds one will need to actually
test and check to fully validate s390.

It depends on the change. If you change the implementation of the ifunc variants and build with the oldest march level, then due to ifunc-impl-list the tests will run all ifunc variants.

You validate the ifunc implementation itself, you need to also check if it
builds on all the variants (even when you are sure that the ifunc variant
itself works). That's what I am trying to avoid, to have multiple possible
glibc build depending of how you configure the compiler.

You always have to build and test glibc with your used compiler-configuration of the target distro. E.g. using z13 as default architecture level set can also introduce other bugs which are not triggered if the test was previously done with zEC12 as als.
A caller of a function is just calling __GI_XYZ. If you change the caller side then you rely on the correct __GI_XYZ symbol.
If you change one existing implementation of XYZ, then the __GI_ mechanism is not touched and you have to test the changed implementation.
If you add a new implementation for a new machine, then you have to extend and test the mechanism. But then you can also afford to build multiple glibcs. If __GI_XYZ exists twice or does not exist, then you'll get a link error without running the testsuite. If __GI_XYZ points to a false implementation, this is really bad. Then you'll either see it with readelf or it fails as you are running on an too old machine. But a change in the ifunc-resolver can also lead to such an issue.

I am not really following your rationale here, the idea is to trade off
between add build complexity to squeeze some performance versus have a
more simple and predictable configure builds.
My point is I would like to have some measurements if this micro-optimization
really pays for the effort of adding even more build and validation
complexity.

Are all the power[4 to X], multiarch and co variants tested on every change?

I would expected so, however I also expect that most distribution just use
default build option (which use ifunc).
The powerpc situation is not really optimal imho and it is an artefact from
how distro usually deployed glibc for different powerpc chips (multiple
builds with loader selecting the path depending of the hwcap value). I really
would like we we move to deprecate and remove the --with-cpu option, so we
have basically just two variants (enable and disable multiarch).
The power4-only multiarch support was motivated mainly because it is the
minimum chip with support for hp-timing. I am not sure it was the best option
back then, but it indeed simplified how to internally handle the minimum ABI
requirements. One option might indeed use a similar strategy to enable disable
hp-timing.h based on compiler support.
Now for powerpc64le I do see that this kind of refactor make sense, since it
uses the old powerpc64 internal organization. Since the minimum ISA requirement
for powerpc64le is 2.07, it does not make sense pack all old ISA optimization
for ifunc variants.

The change from big to little endian was a hard switch between old vs
new approaches. Then it is simple to not pack old stuff.
If you don't have such a hard switch it depends on where glibc is used.
Today the most distros have decided to use zEC12 as default architecture
level set. For e.g. memcpy, there are three ifunc variants which are all
for CPUs older than zEC12. This means currently either the "oldest" (for
__GI_* calls) or "newest" ifunc variant (called from outside libc.so) is
used. The "middle" ifunc variant is never executed. Thus it makes sense
to not pack this "middle" variant. All machines which have installed
this distro can always execute the "newest" variant. The "oldest" is
only needed due to the special __GI_* mechanism.

As next step the distros will decide to use z13 as default architecture
level set. On a distro perspective, this is also a hard switch to not
support older CPUs anymore. Why shall we pack all the old variants?
Once all distros have decided to not support older CPUs anymore a
further step would be to also remove those variants from source. Such a
step could also happen if the kernel / distros decide to not support
compat mode anymore. Then we could remove the complete s390-32 source.

In the end the distro decides the concrete test environment:
versions+patches of kernel/gcc/binutils/..., the CPUs which should be
supported and the used configure flags of all the involved projects.
Thus you have to build and test this concrete environment.

To "fully validate architecture XYZ" it is also not sufficient to run
all ifunc variants on the "newest" machine - someone could have
"changed" an older ifunc variant by using a too new instruction.

If you do a "change in s390 implementations" you have to test it.
What did you mean with "change"?

A change in one ifunc variant needs testing this ifunc variant. The
mechanism with __GI_* is not touched.

Yes, adding a new ifunc variant for a new CPU needs more testing effort.
But if you have a new CPU you should always build and test glibc with
this march level - if you have added a new ifunc variant or not. In the
end it is the same test effort.

A change could also be in binutils regarding handling of local ifunc
(e.g. those I've mentioned which leaded to a segmentation fault in the
linker). I think having ifunc'ed __GI_* symbols is more risky than not
having ifunc'ed __GI_* symbols.

Post by Adhemerval Zanella

Post by Adhemerval Zanella

The patchset also do some clean up and unifies 31 vs 64bit implementations from s390-32 and s390-64 folders. This simplifies maintenance.

I do agree with these kind of changes.

Post by Adhemerval Zanella

Post by Adhemerval Zanella
The gains I see is a slight reduction is loading time (due no ifunc)
and less code side. Is is what is driving you for this change? Does
it worth the extra testing and validation it might require?

The current s390_vx_libc_ifunc macros are not flexible and do not allow future enhancements as they only use the *_c or *_vx variant.

Right, this is one issue which is not really tied to setting up different
build variants depending of the minimum ISA level.

As soon as future ditros will use z13 as default architecture,
vector instructions can be used by default.  But the current __GI_* functions won't use the vector variants.  Instead the fallback is always used for internal calls and the vector variants are always used for external calls via IFUNC.

Some ABI does impose restriction for ifunc with internal hidden symbols
(powerpc32 and i686 if I recall correctly), is it the case for s390
and/or s390x?

On s390 the compiler will generate e.g. a PC32DBL relocation instead of PLT32DBL one and the GOT pointer might not be setup in r12. But the called PLT stub relies on a setup r12.
And on older binutils (unfortunately I'm not sure if it was version >= or < 2.25) bugs regarding ifunc lead to a segmentation fault while linking libc.so.

Do you consider this a deal-break issue? One option is to check the minimum
supported binutils for this change and set it as the minimum required one,
another is disable internal ifunc for such ABI (as for powerpc32, powerpc64
does work).

The "not setuped r12 register" is no binutils-bug. This is such an restriction with ifunc and internal hidden symbols you've mentioned.
Thus I can't use IFUNC __GI_* symbols.
I disable internal ifunc by setting the __GI_* symbols to the "oldest" implementation. Or is there another mechanism?

Is is an issue for both s390 and s390x?

This is an issue for s390 but not for s390x.
Do you think about introducing ifunc'ed __GI_* for s390x and
non-ifunc'ed __GI_* for s390? This will also complicate the __GI_ mechanism.

Post by Adhemerval Zanella

Post by Adhemerval Zanella

Post by Adhemerval Zanella
At least for some ABI, we can still route some internal symbols through
ifunc. On some x86 ifunc selector implementation the comments state it
might show no performance gain, but I am not sure about the validity of
the claims. So if s390 or s390x does not have any impeding reason, a
possibility might to use ifunc internally instead of relying on compiler
default or used options to setup the minimum ABI.

If I start routing those symbols via IFUNC then an additional PLT stub is introduced in libc.so. If we are running on >=z13 then the __XYZ_vx is called via PLT. On machines <z13 __XYZ_c is also called via PLT. Compared to non-ifunc __GI_ symbols we have introduced the extra PLT stub overhead without any advantage on <z13 machines as the __XYZ_c implementation is called.

Do we have real usercases where they are stressing libc symbols that are
doing intra-symbol calls and which PLT overhead is dominant?

Usually we try to avoid PLT calls. I think this is one idea behind the __GI_* symbols. Shall we remove those symbols and always jump over PLT?

It really depends of the implementation that is calling and if it might be
a hotspot. Again I really want to understand if this change is just for the
sake of a micro-optimization or if you really targetting a real usercase
where internal __GI_ symbols are being a hotspot.
In any case, what do you think about reworking the patch to refactor s390
ifunc to be able to handle more variants, unify the code, and add z13
variants and start a separate thread about how to handle the ifunc and
compiler default options? I really want to check with distro focal points
and other maintainers if my objections does indeed does make sense.

Post by Adhemerval Zanella

Post by Adhemerval Zanella
So the question is whether the possible internal symbols call optimization
worth that added complexity for build and validation.

E.g. for strstr there is an advantage. On s390 we are jumping from the common-code implementations to the vector implementations. On other archs we would jump from one vector implementation to another vector implementation and you perhaps don't see so much difference.

One option maybe is maybe to trade some code size for a specific strstr
variant for z13, which calls strstr/strnlen/memcpy locally.  Specially
for strstr, is the difference really worth to add this build and validation
complexity?