diff options
51 files changed, 1029 insertions, 615 deletions
diff --git a/android-changes-for-ndk-developers.md b/android-changes-for-ndk-developers.md index 4bd97a0fe..0fe16bf27 100644 --- a/android-changes-for-ndk-developers.md +++ b/android-changes-for-ndk-developers.md @@ -46,6 +46,10 @@ versions older than JB-MR2, you might want to consider [ReLinker](https://github.com/KeepSafe/ReLinker) which claims to solve these problems automatically. +Alternatively, if you don't have too many dependencies, it can be easiest to +simply link all of your code into one big library and sidestep the details of +library and symbol lookup changes on all past (and future) Android versions. + ## Changes to library search order We have made various fixes to library search order when resolving symbols. diff --git a/docs/native_allocator.md b/docs/native_allocator.md index 97c3648f0..2d06e2f3e 100644 --- a/docs/native_allocator.md +++ b/docs/native_allocator.md @@ -10,7 +10,7 @@ at least the It is important to note that there are two modes for a native allocator to run in on Android. The first is the normal allocator, the second is called the svelte config, which is designed to run on memory constrained -systems and be a bit slower, but take less PSS. To enable the svelte config, +systems and be a bit slower, but take less RSS. To enable the svelte config, add this line to the `BoardConfig.mk` for the given target: MALLOC_SVELTE := true @@ -55,7 +55,7 @@ to work efficiently. When set to zero, `mallopt(M_DECAY_TIME, 0)`, it is expected that an allocator will attempt to purge and release any unused memory back to the kernel on free calls. This is important in Android to avoid consuming extra -PSS. +RSS. When set to non-zero, `mallopt(M_DECAY_TIME, 1)`, an allocator can delay the purge and release action. The amount of delay is up to the allocator @@ -65,13 +65,13 @@ allocator was implemented to have a one second delay. The drawback to this option is that most allocators do not have a separate thread to handle the purge, so the decay is only handled when an allocation operation occurs. For server processes, this can mean that -PSS is slightly higher when the server is waiting for the next connection +RSS is slightly higher when the server is waiting for the next connection and no other allocation calls are made. The `M_PURGE` option is used to force a purge in this case. For all applications on Android, the call `mallopt(M_DECAY_TIME, 1)` is made by default. The idea is that it allows application frees to run a -bit faster, while only increasing PSS a bit. +bit faster, while only increasing RSS a bit. #### M\_PURGE When called, `mallopt(M_PURGE, 0)`, an allocator should purge and release @@ -115,7 +115,7 @@ to specify a device. ## Performance There are multiple different ways to evaluate the performance of a native allocator on Android. One is allocation speed in various different scenarios, -anoher is total PSS taken by the allocator. +another is total RSS taken by the allocator. The last is virtual address space consumed in 32 bit applications. There is a limited amount of address space available in 32 bit apps, and there have @@ -129,7 +129,7 @@ benchmarks. These benchmarks can be built using this command: mmma -j bionic/benchmarks These benchmarks are only used to verify the speed of the allocator and -ignore anything related to PSS and virtual address space consumed. +ignore anything related to RSS and virtual address space consumed. #### Allocate/Free Benchmarks These are the benchmarks to verify the allocation speed of a loop doing a @@ -228,7 +228,7 @@ To run the benchmarks with `mallopt(M_DECAY_TIME, 1)`, use these commands: These numbers should be as performant as the current allocator. ### Memory Trace Benchmarks -These benchmarks measure all three axes of a native allocator, PSS, virtual +These benchmarks measure all three axes of a native allocator, RSS, virtual address space consumed, speed of allocation. They are designed to run on a trace of the allocations from a real world application or system process. @@ -248,7 +248,7 @@ And these two benchmark executables: /data/benchmarktest/trace_benchmark/trace_benchmark #### Memory Replay Benchmarks -These benchmarks display PSS, virtual memory consumed (VA space), and do a +These benchmarks display RSS, virtual memory consumed (VA space), and do a bit of performance testing on actual traces taken from running applications. The trace data includes what thread does each operation, so the replay @@ -279,8 +279,8 @@ Run the benchmark thusly: Where XXX.txt is the name of a trace file. -Every 100000 allocation operations, a dump of the PSS and VA space will be -performed. At the end, a final PSS and VA space number will be printed. +Every 100000 allocation operations, a dump of the RSS and VA space will be +performed. At the end, a final RSS and VA space number will be printed. For the most part, the intermediate data can be ignored, but it is always a good idea to look over the data to verify that no strange spikes are occurring. @@ -304,6 +304,14 @@ important to verify that when running this trace using the 32 bit replay executable, the virtual address space consumed is not much larger than the current allocator. A small increase (on the order of a few MBs) would be okay. +There is no specific benchmark for memory fragmentation, instead, the RSS +when running the memory traces acts as a proxy for this. An allocator that +is fragmenting badly will show an increase in RSS. The best trace for +tracking fragmentation is system\_server.txt which is an extremely long +trace (~13 million operations). The total number of live allocations goes +up and down a bit, but stays mostly the same so an allocator that fragments +badly would likely show an abnormal increase in RSS on this trace. + NOTE: When a native allocator calls mmap, it is expected that the allocator will name the map using the call: diff --git a/docs/status.md b/docs/status.md index 5c2f2e8d4..4466a3c47 100644 --- a/docs/status.md +++ b/docs/status.md @@ -39,6 +39,9 @@ Current libc symbols: https://android.googlesource.com/platform/bionic/+/master/ New libc functions in R (API level 30): * Full C11 `<threads.h>` (available as inlines for older API levels). + * `memfd_create` and `mlock2` (GNU extensions). + * `renameat2` (GNU extension). + * `pthread_cond_clockwait`/`pthread_mutex_clocklock`/`pthread_rwlock_clockrdlock`/`pthread_rwlock_clockwrlock`/`sem_clockwait` New libc functions in Q (API level 29): * `timespec_get` (C11 `<time.h>` addition) diff --git a/libc/Android.bp b/libc/Android.bp index 21ac02e8a..71d4d4a6a 100644 --- a/libc/Android.bp +++ b/libc/Android.bp @@ -1675,7 +1675,9 @@ cc_library { versions: ["10000"], }, - symbol_ordering_file: "symbol_ordering", + // Sorting bss symbols by size usually results in less dirty pages at run + // time, because small symbols are grouped together. + sort_bss_symbols_by_size: true, } genrule { @@ -1710,6 +1712,19 @@ genrule { cmd: "$(location :bionic-generate-version-script) x86_64 $(in) $(out)", } +// Makes bionic_tls.h available for art to use in its implementation of Thread::Current(). +cc_library_headers { + name: "bionic_libc_private_headers", + visibility: [ + "//art:__subpackages__", + ], + host_supported: true, + export_include_dirs: [ + "private", + ], + sdk_version: "current", +} + // libc_headers for libasync_safe and libpropertyinfoparser cc_library_headers { name: "libc_headers", diff --git a/libc/NOTICE b/libc/NOTICE index ef3173387..d9ac638ba 100644 --- a/libc/NOTICE +++ b/libc/NOTICE @@ -1025,6 +1025,22 @@ SUCH DAMAGE. ------------------------------------------------------------------- Copyright (C) 2019 The Android Open Source Project + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +------------------------------------------------------------------- + +Copyright (C) 2019 The Android Open Source Project All rights reserved. Redistribution and use in source and binary forms, with or without @@ -1052,6 +1068,34 @@ SUCH DAMAGE. ------------------------------------------------------------------- +Copyright (C) 2019 The Android Open Source Project +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: +Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. +Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS +FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE +COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, +INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, +BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS +OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED +AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE. + +------------------------------------------------------------------- + Copyright (c) 1980, 1983, 1988, 1993 The Regents of the University of California. All rights reserved. diff --git a/libc/SECCOMP_WHITELIST_COMMON.TXT b/libc/SECCOMP_WHITELIST_COMMON.TXT index 43e7d1c0c..07f84a8fb 100644 --- a/libc/SECCOMP_WHITELIST_COMMON.TXT +++ b/libc/SECCOMP_WHITELIST_COMMON.TXT @@ -64,21 +64,45 @@ int open:open(const char*, int, ...) arm,x86,x86_64 int stat64:stat64(const char*, struct stat64*) arm,x86 ssize_t readlink:readlink(const char*, char*, size_t) arm,x86,x86_64 -# Probed for and conditionally used by ART. -int membarrier(int cmd, int flags) all - +# # Useful new syscalls which we don't yet use in bionic. +# + +# Since Linux 2.5, not in glibc. +int io_setup(unsigned nr, aio_context_t *ctxp) all +int io_destroy(aio_context_t ctx) all +int io_submit(aio_context_t ctx, long nr, struct iocb **iocbpp) all +int io_getevents(aio_context_t ctx, long min_nr, long max_nr, struct io_event *events, struct timespec *timeout) all +int io_cancel(aio_context_t ctx, struct iocb *, struct io_event *result) all +# Since Linux 3.14, not in glibc. int sched_getattr(pid_t pid, struct sched_attr* attr, unsigned int flags) all int sched_setattr(pid_t pid, struct sched_attr* attr, unsigned int size, unsigned int flags) all -int memfd_create(const char* name, unsigned int flags) all -int renameat2(int olddirfd, const char* oldpath, int newdirfd, const char* newpath, unsigned int flags) all +# Since Linux 3.19, not in glibc (and not really needed to implement fexecve). int execveat(int dirfd, const char* pathname, char* const* argv, char* const* envp, int flags) all +# Since Linux 4.3, not in glibc. Probed for and conditionally used by ART. +int membarrier(int cmd, int flags) all +# Since Linux 4.5, glibc 2.27. ssize_t copy_file_range(int fd_in, loff_t* off_in, int fd_out, loff_t* off_out, size_t len, unsigned int flags) all -int mlock2(const void* addr, size_t len, int flags) all +# Since Linux 4.6, glibc 2.26. ssize_t preadv2(int fd, const struct iovec* iov, int iovcnt, off_t offset, int flags) all ssize_t pwritev2(int fd, const struct iovec* iov, int iovcnt, off_t offset, int flags) all -int io_setup(unsigned nr, aio_context_t *ctxp) all -int io_destroy(aio_context_t ctx) all -int io_submit(aio_context_t ctx, long nr, struct iocb **iocbpp) all -int io_getevents(aio_context_t ctx, long min_nr, long max_nr, struct io_event *events, struct timespec *timeout) all -int io_cancel(aio_context_t ctx, struct iocb *, struct io_event *result) all +# Since Linux 4.11, glibc 2.30. +int statx(int, const char*, int, unsigned int, statx*) all +# Since Linux 5.1, not in glibc. +int clock_gettime64(clockid_t, timespec64*) lp32 +int clock_settime64(clockid_t, const timespec64*) lp32 +int clock_adjtime64(clockid_t, timex64*) lp32 +int clock_getres_time64(clockid_t, timespec64*) lp32 +int clock_nanosleep_time64(clockid_t, int, const timespec64*, timespec*) lp32 +int timer_gettime64(__kernel_timer_t, itimerspec64*) lp32 +int timer_settime64(__kernel_timer_t, int, const itimerspec64*, itimerspec64*) lp32 +int timerfd_gettime64(int, itimerspec64*) lp32 +int timerfd_settime64(int, int, const itimerspec64*, itimerspec64*) lp32 +int utimensat_time64(int, const char*, const timespec64[2], int) lp32 +int pselect6_time64(int, fd_set*, fd_set*, timespec64*, void*) lp32 +int ppoll_time64(pollfd*, unsigned int, timespec64*, const sigset64_t*, size_t) lp32 +int recvmmsg_time64(int, mmsghdr*, unsigned int, int, const timespec64*) lp32 +int semtimedop_time64(int, sembuf*, size_t, const timespec64*) lp32 +int rt_sigtimedwait_time64(const sigset64_t*, siginfo_t*, const timespec64*, size_t) lp32 +int futex_time64(int*, int, int, const timespec64*, int*, int) lp32 +int sched_rr_get_interval_time64(pid_t, timespec64*) lp32 diff --git a/libc/SYSCALLS.TXT b/libc/SYSCALLS.TXT index 99bdc934e..21ebdbd8e 100644 --- a/libc/SYSCALLS.TXT +++ b/libc/SYSCALLS.TXT @@ -107,12 +107,14 @@ ssize_t pwritev|pwritev64(int, const struct iovec*, int, off_t) lp64 int __close:close(int) all pid_t __getpid:getpid() all +int memfd_create(const char*, unsigned) all int munmap(void*, size_t) all void* __mremap:mremap(void*, size_t, size_t, int, void*) all int msync(const void*, size_t, int) all int mprotect(const void*, size_t, int) all int madvise(void*, size_t, int) all -int mlock(const void* addr, size_t len) all +int mlock(const void* addr, size_t len) all +int mlock2(const void* addr, size_t len, int flags) all int munlock(const void* addr, size_t len) all int mlockall(int flags) all int munlockall() all @@ -151,6 +153,7 @@ int mkdirat(int, const char*, mode_t) all int mknodat(int, const char*, mode_t, dev_t) all int readlinkat(int, const char*, char*, size_t) all int renameat(int, const char*, int, const char*) all +int renameat2(int, const char*, int, const char*, unsigned) all int symlinkat(const char*, int, const char*) all int unlinkat(int, const char*, int) all int utimensat(int, const char*, const struct timespec times[2], int) all diff --git a/libc/arch-x86/generic/string/wmemset.c b/libc/arch-x86/generic/string/wmemset.c index 6247bfd05..eff533c10 100644 --- a/libc/arch-x86/generic/string/wmemset.c +++ b/libc/arch-x86/generic/string/wmemset.c @@ -1,16 +1,18 @@ -// Copyright (C) 2019 The Android Open Source Project -// -// Licensed under the Apache License, Version 2.0 (the "License"); -// you may not use this file except in compliance with the License. -// You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. +/* + * Copyright (C) 2019 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. +*/ #define wmemset wmemset_freebsd diff --git a/libc/bionic/__cxa_guard.cpp b/libc/bionic/__cxa_guard.cpp index 30b5f41bd..e2e747791 100644 --- a/libc/bionic/__cxa_guard.cpp +++ b/libc/bionic/__cxa_guard.cpp @@ -16,9 +16,7 @@ #include <endian.h> #include <limits.h> -#undef _USING_LIBCXX // Prevent using of <atomic>. #include <stdatomic.h> - #include <stddef.h> #include "private/bionic_futex.h" diff --git a/libc/bionic/__libc_init_main_thread.cpp b/libc/bionic/__libc_init_main_thread.cpp index 4984e38a1..6e1b0de25 100644 --- a/libc/bionic/__libc_init_main_thread.cpp +++ b/libc/bionic/__libc_init_main_thread.cpp @@ -135,8 +135,11 @@ extern "C" void __libc_init_main_thread_final() { main_thread.mmap_base = mapping.mmap_base; main_thread.mmap_size = mapping.mmap_size; + main_thread.mmap_base_unguarded = mapping.mmap_base_unguarded; + main_thread.mmap_size_unguarded = mapping.mmap_size_unguarded; __set_tls(&new_tcb->tls_slot(0)); + __set_stack_and_tls_vma_name(true); __free_temp_bionic_tls(temp_tls); } diff --git a/libc/bionic/fork.cpp b/libc/bionic/fork.cpp index fc0020798..cda5e8533 100644 --- a/libc/bionic/fork.cpp +++ b/libc/bionic/fork.cpp @@ -55,6 +55,10 @@ int fork() { // fork, close all of their fds blindly, and then exec. android_fdsan_set_error_level(ANDROID_FDSAN_ERROR_LEVEL_DISABLED); + // Reset the stack_and_tls VMA name so it doesn't end with a tid from the + // parent process. + __set_stack_and_tls_vma_name(true); + __bionic_atfork_run_child(); } else { __bionic_atfork_run_parent(); diff --git a/libc/bionic/grp_pwd.cpp b/libc/bionic/grp_pwd.cpp index d37fadd38..6f8c07e48 100644 --- a/libc/bionic/grp_pwd.cpp +++ b/libc/bionic/grp_pwd.cpp @@ -50,8 +50,21 @@ #include "generated_android_ids.h" #include "grp_pwd_file.h" -static PasswdFile vendor_passwd("/vendor/etc/passwd", "vendor_"); -static GroupFile vendor_group("/vendor/etc/group", "vendor_"); +static PasswdFile passwd_files[] = { + { "/system/etc/passwd", "system_" }, + { "/vendor/etc/passwd", "vendor_" }, + { "/odm/etc/passwd", "odm_" }, + { "/product/etc/passwd", "product_" }, + { "/system_ext/etc/passwd", "system_ext_" }, +}; + +static GroupFile group_files[] = { + { "/system/etc/group", "system_" }, + { "/vendor/etc/group", "vendor_" }, + { "/odm/etc/group", "odm_" }, + { "/product/etc/group", "product_" }, + { "/system_ext/etc/group", "system_ext_" }, +}; // POSIX seems to envisage an implementation where the <pwd.h> functions are // implemented by brute-force searching with getpwent(3), and the <grp.h> @@ -91,7 +104,7 @@ static passwd* android_iinfo_to_passwd(passwd_state_t* state, const android_id_info* iinfo) { snprintf(state->name_buffer_, sizeof(state->name_buffer_), "%s", iinfo->name); snprintf(state->dir_buffer_, sizeof(state->dir_buffer_), "/"); - snprintf(state->sh_buffer_, sizeof(state->sh_buffer_), "/system/bin/sh"); + snprintf(state->sh_buffer_, sizeof(state->sh_buffer_), "/bin/sh"); passwd* pw = &state->passwd_; pw->pw_uid = iinfo->aid; @@ -421,17 +434,19 @@ static id_t oem_id_from_name(const char* name) { } static passwd* oem_id_to_passwd(uid_t uid, passwd_state_t* state) { - if (!is_oem_id(uid)) { - return nullptr; + for (auto& passwd_file : passwd_files) { + if (passwd_file.FindById(uid, state)) { + return &state->passwd_; + } } - if (vendor_passwd.FindById(uid, state)) { - return &state->passwd_; + if (!is_oem_id(uid)) { + return nullptr; } snprintf(state->name_buffer_, sizeof(state->name_buffer_), "oem_%u", uid); snprintf(state->dir_buffer_, sizeof(state->dir_buffer_), "/"); - snprintf(state->sh_buffer_, sizeof(state->sh_buffer_), "/vendor/bin/sh"); + snprintf(state->sh_buffer_, sizeof(state->sh_buffer_), "/bin/sh"); passwd* pw = &state->passwd_; pw->pw_uid = uid; @@ -440,12 +455,14 @@ static passwd* oem_id_to_passwd(uid_t uid, passwd_state_t* state) { } static group* oem_id_to_group(gid_t gid, group_state_t* state) { - if (!is_oem_id(gid)) { - return nullptr; + for (auto& group_file : group_files) { + if (group_file.FindById(gid, state)) { + return &state->group_; + } } - if (vendor_group.FindById(gid, state)) { - return &state->group_; + if (!is_oem_id(gid)) { + return nullptr; } snprintf(state->group_name_buffer_, sizeof(state->group_name_buffer_), @@ -477,7 +494,7 @@ static passwd* app_id_to_passwd(uid_t uid, passwd_state_t* state) { snprintf(state->dir_buffer_, sizeof(state->dir_buffer_), "/data"); } - snprintf(state->sh_buffer_, sizeof(state->sh_buffer_), "/system/bin/sh"); + snprintf(state->sh_buffer_, sizeof(state->sh_buffer_), "/bin/sh"); passwd* pw = &state->passwd_; pw->pw_uid = uid; @@ -523,8 +540,8 @@ passwd* getpwnam_internal(const char* login, passwd_state_t* state) { return android_iinfo_to_passwd(state, android_id_info); } - if (vendor_passwd.FindByName(login, state)) { - if (is_oem_id(state->passwd_.pw_uid)) { + for (auto& passwd_file : passwd_files) { + if (passwd_file.FindByName(login, state)) { return &state->passwd_; } } @@ -633,6 +650,22 @@ passwd* getpwent() { state->getpwent_idx++ - start + AID_OEM_RESERVED_2_START, state); } + start = end; + end += AID_SYSTEM_EXT_RESERVED_END - AID_SYSTEM_RESERVED_START + 1; + + if (state->getpwent_idx < end) { + // No one calls this enough to worry about how inefficient the below is. + auto* oem_passwd = + oem_id_to_passwd(state->getpwent_idx++ - start + AID_SYSTEM_RESERVED_START, state); + while (oem_passwd == nullptr && state->getpwent_idx < end) { + oem_passwd = + oem_id_to_passwd(state->getpwent_idx++ - start + AID_SYSTEM_RESERVED_START, state); + } + if (oem_passwd != nullptr) { + return oem_passwd; + } + } + state->getpwent_idx = get_next_app_id(state->getpwent_idx, false); if (state->getpwent_idx != -1) { @@ -666,8 +699,8 @@ static group* getgrnam_internal(const char* name, group_state_t* state) { return android_iinfo_to_group(state, android_id_info); } - if (vendor_group.FindByName(name, state)) { - if (is_oem_id(state->group_.gr_gid)) { + for (auto& group_file : group_files) { + if (group_file.FindByName(name, state)) { return &state->group_; } } @@ -754,6 +787,22 @@ group* getgrent() { } start = end; + end += AID_SYSTEM_EXT_RESERVED_END - AID_SYSTEM_RESERVED_START + 1; + + if (state->getgrent_idx < end) { + // No one calls this enough to worry about how inefficient the below is. + init_group_state(state); + auto* oem_group = + oem_id_to_group(state->getgrent_idx++ - start + AID_SYSTEM_RESERVED_START, state); + while (oem_group == nullptr && state->getgrent_idx < end) { + oem_group = oem_id_to_group(state->getgrent_idx++ - start + AID_SYSTEM_RESERVED_START, state); + } + if (oem_group != nullptr) { + return oem_group; + } + } + + start = end; end += AID_USER_OFFSET - AID_APP_START; // Do not expose higher groups state->getgrent_idx = get_next_app_id(state->getgrent_idx, true); diff --git a/libc/bionic/grp_pwd_file.cpp b/libc/bionic/grp_pwd_file.cpp index 201c9d084..e13604e68 100644 --- a/libc/bionic/grp_pwd_file.cpp +++ b/libc/bionic/grp_pwd_file.cpp @@ -270,8 +270,8 @@ bool MmapFile::Find(Line* line, Predicate predicate) { while (line_beginning < end) { line_beginning = ParseLine(line_beginning, end, line->fields, line->kNumFields); - // To comply with Treble, users/groups from the vendor partition need to be prefixed with - // vendor_. + // To comply with Treble, users/groups from each partition need to be prefixed with + // the partition name. if (required_prefix_ != nullptr) { if (strncmp(line->fields[0], required_prefix_, strlen(required_prefix_)) != 0) { char name[kGrpPwdBufferSize]; diff --git a/libc/bionic/jemalloc.h b/libc/bionic/jemalloc.h index b9a4e993c..ef77c9c99 100644 --- a/libc/bionic/jemalloc.h +++ b/libc/bionic/jemalloc.h @@ -30,7 +30,7 @@ __BEGIN_DECLS void* je_aligned_alloc_wrapper(size_t, size_t); int je_iterate(uintptr_t, size_t, void (*)(uintptr_t, size_t, void*), void*); -int je_mallctl(const char *name, void *oldp, size_t *oldlenp, void *newp, size_t newlen); +int je_mallctl(const char *name, void *oldp, size_t *oldlenp, void *newp, size_t newlen) __attribute__((nothrow)); struct mallinfo je_mallinfo(); void je_malloc_disable(); void je_malloc_enable(); diff --git a/libc/bionic/libc_init_dynamic.cpp b/libc/bionic/libc_init_dynamic.cpp index 4f3b4f7d1..d6d5552d1 100644 --- a/libc/bionic/libc_init_dynamic.cpp +++ b/libc/bionic/libc_init_dynamic.cpp @@ -69,6 +69,13 @@ extern "C" { __LIBC_HIDDEN__ void* __libc_sysinfo = reinterpret_cast<void*>(__libc_int0x80); #endif +extern "C" __attribute__((weak)) void __hwasan_library_loaded(ElfW(Addr) base, + const ElfW(Phdr)* phdr, + ElfW(Half) phnum); +extern "C" __attribute__((weak)) void __hwasan_library_unloaded(ElfW(Addr) base, + const ElfW(Phdr)* phdr, + ElfW(Half) phnum); + // We need a helper function for __libc_preinit because compiling with LTO may // inline functions requiring a stack protector check, but __stack_chk_guard is // not initialized at the start of __libc_preinit. __libc_preinit_impl will run @@ -91,6 +98,14 @@ static void __libc_preinit_impl() { // Hooks for various libraries to let them know that we're starting up. __libc_globals.mutate(__libc_init_malloc); + +#if __has_feature(hwaddress_sanitizer) + // Notify the HWASan runtime library whenever a library is loaded or unloaded + // so that it can update its shadow memory. + __libc_shared_globals()->load_hook = __hwasan_library_loaded; + __libc_shared_globals()->unload_hook = __hwasan_library_unloaded; +#endif + netdClientInit(); } diff --git a/libc/bionic/pthread_cond.cpp b/libc/bionic/pthread_cond.cpp index 1ab1b81c4..e85706902 100644 --- a/libc/bionic/pthread_cond.cpp +++ b/libc/bionic/pthread_cond.cpp @@ -221,7 +221,7 @@ int pthread_cond_clockwait(pthread_cond_t* cond_interface, pthread_mutex_t* mute case CLOCK_MONOTONIC: return pthread_cond_timedwait_monotonic_np(cond_interface, mutex, abs_timeout); case CLOCK_REALTIME: - return pthread_cond_timedwait(cond_interface, mutex, abs_timeout); + return __pthread_cond_timedwait(__get_internal_cond(cond_interface), mutex, true, abs_timeout); default: return EINVAL; } diff --git a/libc/bionic/pthread_create.cpp b/libc/bionic/pthread_create.cpp index 4cf14ad72..0dbf53949 100644 --- a/libc/bionic/pthread_create.cpp +++ b/libc/bionic/pthread_create.cpp @@ -254,6 +254,8 @@ ThreadMapping __allocate_thread_mapping(size_t stack_size, size_t stack_guard_si ThreadMapping result = {}; result.mmap_base = space; result.mmap_size = mmap_size; + result.mmap_base_unguarded = space + stack_guard_size; + result.mmap_size_unguarded = mmap_size - stack_guard_size - PTHREAD_GUARD_SIZE; result.static_tls = space + mmap_size - PTHREAD_GUARD_SIZE - layout.size(); result.stack_base = space; result.stack_top = result.static_tls; @@ -315,12 +317,33 @@ static int __allocate_thread(pthread_attr_t* attr, bionic_tcb** tcbp, void** chi thread->attr = *attr; thread->mmap_base = mapping.mmap_base; thread->mmap_size = mapping.mmap_size; + thread->mmap_base_unguarded = mapping.mmap_base_unguarded; + thread->mmap_size_unguarded = mapping.mmap_size_unguarded; *tcbp = tcb; *child_stack = stack_top; return 0; } +void __set_stack_and_tls_vma_name(bool is_main_thread) { + // Name the thread's stack-and-tls area to help with debugging. This mapped area also includes + // static TLS data, which is typically a few pages (e.g. bionic_tls). + pthread_internal_t* thread = __get_thread(); + const char* name; + if (is_main_thread) { + name = "stack_and_tls:main"; + } else { + // The kernel doesn't copy the name string, but this variable will last at least as long as the + // mapped area. The mapped area's VMAs are unmapped with a single call to munmap. + auto& name_buffer = thread->vma_name_buffer; + static_assert(arraysize(name_buffer) >= arraysize("stack_and_tls:") + 11 + 1); + async_safe_format_buffer(name_buffer, arraysize(name_buffer), "stack_and_tls:%d", thread->tid); + name = name_buffer; + } + prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, thread->mmap_base_unguarded, thread->mmap_size_unguarded, + name); +} + __attribute__((no_sanitize("hwaddress"))) static int __pthread_start(void* arg) { pthread_internal_t* thread = reinterpret_cast<pthread_internal_t*>(arg); @@ -333,6 +356,7 @@ static int __pthread_start(void* arg) { // accesses previously made by the creating thread are visible to us. thread->startup_handshake_lock.lock(); + __set_stack_and_tls_vma_name(false); __init_additional_stacks(thread); void* result = thread->start_routine(thread->start_routine_arg); diff --git a/libc/bionic/pthread_internal.h b/libc/bionic/pthread_internal.h index a1e0c450a..22cc4000d 100644 --- a/libc/bionic/pthread_internal.h +++ b/libc/bionic/pthread_internal.h @@ -129,6 +129,11 @@ class pthread_internal_t { void* mmap_base; size_t mmap_size; + // The location of the VMA to label as the thread's stack_and_tls. + void* mmap_base_unguarded; + size_t mmap_size_unguarded; + char vma_name_buffer[32]; + thread_local_dtor* thread_local_dtors; /* @@ -147,6 +152,8 @@ class pthread_internal_t { struct ThreadMapping { char* mmap_base; size_t mmap_size; + char* mmap_base_unguarded; + size_t mmap_size_unguarded; char* static_tls; char* stack_base; @@ -162,6 +169,7 @@ __LIBC_HIDDEN__ void __free_temp_bionic_tls(bionic_tls* tls); __LIBC_HIDDEN__ void __init_additional_stacks(pthread_internal_t*); __LIBC_HIDDEN__ int __init_thread(pthread_internal_t* thread); __LIBC_HIDDEN__ ThreadMapping __allocate_thread_mapping(size_t stack_size, size_t stack_guard_size); +__LIBC_HIDDEN__ void __set_stack_and_tls_vma_name(bool is_main_thread); __LIBC_HIDDEN__ pthread_t __pthread_internal_add(pthread_internal_t* thread); __LIBC_HIDDEN__ pthread_internal_t* __pthread_internal_find(pthread_t pthread_id, const char* caller); diff --git a/libc/include/android/legacy_signal_inlines.h b/libc/include/android/legacy_signal_inlines.h index 5ca9813f3..d2c767703 100644 --- a/libc/include/android/legacy_signal_inlines.h +++ b/libc/include/android/legacy_signal_inlines.h @@ -52,7 +52,7 @@ static __inline int __ndk_legacy___libc_current_sigrtmax() { static __inline int __ndk_legacy___libc_current_sigrtmin() { if (__libc_current_sigrtmin) return __libc_current_sigrtmin(); - return __SIGRTMIN + 6; /* Should match __libc_current_sigrtmin. */ + return __SIGRTMIN + 7; /* Should match __libc_current_sigrtmin. */ } #undef SIGRTMAX diff --git a/libc/include/bits/elf_arm64.h b/libc/include/bits/elf_arm64.h index d838c0b2d..6bb8384e4 100644 --- a/libc/include/bits/elf_arm64.h +++ b/libc/include/bits/elf_arm64.h @@ -83,9 +83,9 @@ #define R_AARCH64_GLOB_DAT 1025 /* Create GOT entry. */ #define R_AARCH64_JUMP_SLOT 1026 /* Create PLT entry. */ #define R_AARCH64_RELATIVE 1027 /* Adjust by program base. */ -#define R_AARCH64_TLS_DTPREL64 1028 /* Module-relative offset. */ -#define R_AARCH64_TLS_DTPMOD64 1029 /* Module index. */ -#define R_AARCH64_TLS_TPREL64 1030 /* TP-relative offset. */ +#define R_AARCH64_TLS_DTPMOD 1028 /* Module index. */ +#define R_AARCH64_TLS_DTPREL 1029 /* Module-relative offset. */ +#define R_AARCH64_TLS_TPREL 1030 /* TP-relative offset. */ #define R_AARCH64_TLSDESC 1031 /* 16-byte descriptor: resolver func + arg. */ #define R_AARCH64_IRELATIVE 1032 diff --git a/libc/include/bits/stdatomic.h b/libc/include/bits/stdatomic.h new file mode 100644 index 000000000..633cb8665 --- /dev/null +++ b/libc/include/bits/stdatomic.h @@ -0,0 +1,286 @@ +/*- + * Copyright (c) 2011 Ed Schouten <ed@FreeBSD.org> + * David Chisnall <theraven@FreeBSD.org> + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#pragma once + +#include <sys/cdefs.h> +#include <sys/types.h> +#include <stdbool.h> + +/* + * C: Do it ourselves. + * Note that the runtime representation defined here should be compatible + * with the C++ one, i.e. an _Atomic(T) needs to contain the same + * bits as a T. + */ + +#include <stddef.h> /* For ptrdiff_t. */ +#include <stdint.h> /* TODO: don't drag in all the macros, just the types. */ +// Include uchar.h only when available. Bionic's stdatomic.h is also used for +// the host (via a copy in prebuilts/clang) and uchar.h is not available in the +// glibc used for the host. +#if defined(__BIONIC__) +# include <uchar.h> /* For char16_t and char32_t. */ +#endif + +/* + * 7.17.1 Atomic lock-free macros. + */ + +#ifdef __GCC_ATOMIC_BOOL_LOCK_FREE +#define ATOMIC_BOOL_LOCK_FREE __GCC_ATOMIC_BOOL_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_CHAR_LOCK_FREE +#define ATOMIC_CHAR_LOCK_FREE __GCC_ATOMIC_CHAR_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_CHAR16_T_LOCK_FREE +#define ATOMIC_CHAR16_T_LOCK_FREE __GCC_ATOMIC_CHAR16_T_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_CHAR32_T_LOCK_FREE +#define ATOMIC_CHAR32_T_LOCK_FREE __GCC_ATOMIC_CHAR32_T_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_WCHAR_T_LOCK_FREE +#define ATOMIC_WCHAR_T_LOCK_FREE __GCC_ATOMIC_WCHAR_T_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_SHORT_LOCK_FREE +#define ATOMIC_SHORT_LOCK_FREE __GCC_ATOMIC_SHORT_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_INT_LOCK_FREE +#define ATOMIC_INT_LOCK_FREE __GCC_ATOMIC_INT_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_LONG_LOCK_FREE +#define ATOMIC_LONG_LOCK_FREE __GCC_ATOMIC_LONG_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_LLONG_LOCK_FREE +#define ATOMIC_LLONG_LOCK_FREE __GCC_ATOMIC_LLONG_LOCK_FREE +#endif +#ifdef __GCC_ATOMIC_POINTER_LOCK_FREE +#define ATOMIC_POINTER_LOCK_FREE __GCC_ATOMIC_POINTER_LOCK_FREE +#endif + +/* + * 7.17.2 Initialization. + */ + +#define ATOMIC_VAR_INIT(value) (value) +#define atomic_init(obj, value) __c11_atomic_init(obj, value) + +/* + * Clang and recent GCC both provide predefined macros for the memory + * orderings. If we are using a compiler that doesn't define them, use the + * clang values - these will be ignored in the fallback path. + */ + +#ifndef __ATOMIC_RELAXED +#define __ATOMIC_RELAXED 0 +#endif +#ifndef __ATOMIC_CONSUME +#define __ATOMIC_CONSUME 1 +#endif +#ifndef __ATOMIC_ACQUIRE +#define __ATOMIC_ACQUIRE 2 +#endif +#ifndef __ATOMIC_RELEASE +#define __ATOMIC_RELEASE 3 +#endif +#ifndef __ATOMIC_ACQ_REL +#define __ATOMIC_ACQ_REL 4 +#endif +#ifndef __ATOMIC_SEQ_CST +#define __ATOMIC_SEQ_CST 5 +#endif + +/* + * 7.17.3 Order and consistency. + * + * The memory_order_* constants that denote the barrier behaviour of the + * atomic operations. + * The enum values must be identical to those used by the + * C++ <atomic> header. + */ + +typedef enum { + memory_order_relaxed = __ATOMIC_RELAXED, + memory_order_consume = __ATOMIC_CONSUME, + memory_order_acquire = __ATOMIC_ACQUIRE, + memory_order_release = __ATOMIC_RELEASE, + memory_order_acq_rel = __ATOMIC_ACQ_REL, + memory_order_seq_cst = __ATOMIC_SEQ_CST +} memory_order; + +/* + * 7.17.4 Fences. + */ + +static __inline void atomic_thread_fence(memory_order __order __attribute__((unused))) { + __c11_atomic_thread_fence(__order); +} + +static __inline void atomic_signal_fence(memory_order __order __attribute__((unused))) { + __c11_atomic_signal_fence(__order); +} + +/* + * 7.17.5 Lock-free property. + */ + +#define atomic_is_lock_free(obj) __c11_atomic_is_lock_free(sizeof(*(obj))) + +/* + * 7.17.6 Atomic integer types. + */ + +typedef _Atomic(bool) atomic_bool; +typedef _Atomic(char) atomic_char; +typedef _Atomic(signed char) atomic_schar; +typedef _Atomic(unsigned char) atomic_uchar; +typedef _Atomic(short) atomic_short; +typedef _Atomic(unsigned short) atomic_ushort; +typedef _Atomic(int) atomic_int; +typedef _Atomic(unsigned int) atomic_uint; +typedef _Atomic(long) atomic_long; +typedef _Atomic(unsigned long) atomic_ulong; +typedef _Atomic(long long) atomic_llong; +typedef _Atomic(unsigned long long) atomic_ullong; +#if defined(__BIONIC__) || (defined(__cplusplus) && __cplusplus >= 201103L) + typedef _Atomic(char16_t) atomic_char16_t; + typedef _Atomic(char32_t) atomic_char32_t; +#endif +typedef _Atomic(wchar_t) atomic_wchar_t; +typedef _Atomic(int_least8_t) atomic_int_least8_t; +typedef _Atomic(uint_least8_t) atomic_uint_least8_t; +typedef _Atomic(int_least16_t) atomic_int_least16_t; +typedef _Atomic(uint_least16_t) atomic_uint_least16_t; +typedef _Atomic(int_least32_t) atomic_int_least32_t; +typedef _Atomic(uint_least32_t) atomic_uint_least32_t; +typedef _Atomic(int_least64_t) atomic_int_least64_t; +typedef _Atomic(uint_least64_t) atomic_uint_least64_t; +typedef _Atomic(int_fast8_t) atomic_int_fast8_t; +typedef _Atomic(uint_fast8_t) atomic_uint_fast8_t; +typedef _Atomic(int_fast16_t) atomic_int_fast16_t; +typedef _Atomic(uint_fast16_t) atomic_uint_fast16_t; +typedef _Atomic(int_fast32_t) atomic_int_fast32_t; +typedef _Atomic(uint_fast32_t) atomic_uint_fast32_t; +typedef _Atomic(int_fast64_t) atomic_int_fast64_t; +typedef _Atomic(uint_fast64_t) atomic_uint_fast64_t; +typedef _Atomic(intptr_t) atomic_intptr_t; +typedef _Atomic(uintptr_t) atomic_uintptr_t; +typedef _Atomic(size_t) atomic_size_t; +typedef _Atomic(ptrdiff_t) atomic_ptrdiff_t; +typedef _Atomic(intmax_t) atomic_intmax_t; +typedef _Atomic(uintmax_t) atomic_uintmax_t; + +/* + * 7.17.7 Operations on atomic types. + */ + +/* + * Compiler-specific operations. + */ + +#define atomic_compare_exchange_strong_explicit(object, expected, \ + desired, success, failure) \ + __c11_atomic_compare_exchange_strong(object, expected, desired, \ + success, failure) +#define atomic_compare_exchange_weak_explicit(object, expected, \ + desired, success, failure) \ + __c11_atomic_compare_exchange_weak(object, expected, desired, \ + success, failure) +#define atomic_exchange_explicit(object, desired, order) \ + __c11_atomic_exchange(object, desired, order) +#define atomic_fetch_add_explicit(object, operand, order) \ + __c11_atomic_fetch_add(object, operand, order) +#define atomic_fetch_and_explicit(object, operand, order) \ + __c11_atomic_fetch_and(object, operand, order) +#define atomic_fetch_or_explicit(object, operand, order) \ + __c11_atomic_fetch_or(object, operand, order) +#define atomic_fetch_sub_explicit(object, operand, order) \ + __c11_atomic_fetch_sub(object, operand, order) +#define atomic_fetch_xor_explicit(object, operand, order) \ + __c11_atomic_fetch_xor(object, operand, order) +#define atomic_load_explicit(object, order) \ + __c11_atomic_load(object, order) +#define atomic_store_explicit(object, desired, order) \ + __c11_atomic_store(object, desired, order) + +/* + * Convenience functions. + */ + +#define atomic_compare_exchange_strong(object, expected, desired) \ + atomic_compare_exchange_strong_explicit(object, expected, \ + desired, memory_order_seq_cst, memory_order_seq_cst) +#define atomic_compare_exchange_weak(object, expected, desired) \ + atomic_compare_exchange_weak_explicit(object, expected, \ + desired, memory_order_seq_cst, memory_order_seq_cst) +#define atomic_exchange(object, desired) \ + atomic_exchange_explicit(object, desired, memory_order_seq_cst) +#define atomic_fetch_add(object, operand) \ + atomic_fetch_add_explicit(object, operand, memory_order_seq_cst) +#define atomic_fetch_and(object, operand) \ + atomic_fetch_and_explicit(object, operand, memory_order_seq_cst) +#define atomic_fetch_or(object, operand) \ + atomic_fetch_or_explicit(object, operand, memory_order_seq_cst) +#define atomic_fetch_sub(object, operand) \ + atomic_fetch_sub_explicit(object, operand, memory_order_seq_cst) +#define atomic_fetch_xor(object, operand) \ + atomic_fetch_xor_explicit(object, operand, memory_order_seq_cst) +#define atomic_load(object) \ + atomic_load_explicit(object, memory_order_seq_cst) +#define atomic_store(object, desired) \ + atomic_store_explicit(object, desired, memory_order_seq_cst) + +/* + * 7.17.8 Atomic flag type and operations. + * + * XXX: Assume atomic_bool can be used as an atomic_flag. Is there some + * kind of compiler built-in type we could use? + */ + +typedef struct { + atomic_bool __flag; +} atomic_flag; + +#define ATOMIC_FLAG_INIT { ATOMIC_VAR_INIT(false) } + +static __inline bool atomic_flag_test_and_set_explicit(volatile atomic_flag *__object, memory_order __order) { + return (atomic_exchange_explicit(&__object->__flag, 1, __order)); +} + +static __inline void atomic_flag_clear_explicit(volatile atomic_flag *__object, memory_order __order) { + atomic_store_explicit(&__object->__flag, 0, __order); +} + +static __inline bool atomic_flag_test_and_set(volatile atomic_flag *__object) { + return (atomic_flag_test_and_set_explicit(__object, memory_order_seq_cst)); +} + +static __inline void atomic_flag_clear(volatile atomic_flag *__object) { + atomic_flag_clear_explicit(__object, memory_order_seq_cst); +} diff --git a/libc/include/stdatomic.h b/libc/include/stdatomic.h index 3c613346d..b7dac4a44 100644 --- a/libc/include/stdatomic.h +++ b/libc/include/stdatomic.h @@ -32,7 +32,7 @@ #include <sys/cdefs.h> -#if defined(__cplusplus) && __cplusplus >= 201103L && defined(_USING_LIBCXX) +#if defined(__cplusplus) && __cplusplus >= 201103L && __has_include(<atomic>) # if __has_feature(cxx_atomic) # define _STDATOMIC_HAVE_ATOMIC # endif @@ -127,260 +127,8 @@ using std::atomic_uintmax_t; #else /* <atomic> unavailable, possibly because this is C, not C++ */ -#include <sys/types.h> -#include <stdbool.h> - -/* - * C: Do it ourselves. - * Note that the runtime representation defined here should be compatible - * with the C++ one, i.e. an _Atomic(T) needs to contain the same - * bits as a T. - */ - -#include <stddef.h> /* For ptrdiff_t. */ -#include <stdint.h> /* TODO: don't drag in all the macros, just the types. */ -// Include uchar.h only when available. Bionic's stdatomic.h is also used for -// the host (via a copy in prebuilts/clang) and uchar.h is not available in the -// glibc used for the host. -#if defined(__BIONIC__) -# include <uchar.h> /* For char16_t and char32_t. */ -#endif - -/* - * 7.17.1 Atomic lock-free macros. - */ - -#ifdef __GCC_ATOMIC_BOOL_LOCK_FREE -#define ATOMIC_BOOL_LOCK_FREE __GCC_ATOMIC_BOOL_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_CHAR_LOCK_FREE -#define ATOMIC_CHAR_LOCK_FREE __GCC_ATOMIC_CHAR_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_CHAR16_T_LOCK_FREE -#define ATOMIC_CHAR16_T_LOCK_FREE __GCC_ATOMIC_CHAR16_T_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_CHAR32_T_LOCK_FREE -#define ATOMIC_CHAR32_T_LOCK_FREE __GCC_ATOMIC_CHAR32_T_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_WCHAR_T_LOCK_FREE -#define ATOMIC_WCHAR_T_LOCK_FREE __GCC_ATOMIC_WCHAR_T_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_SHORT_LOCK_FREE -#define ATOMIC_SHORT_LOCK_FREE __GCC_ATOMIC_SHORT_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_INT_LOCK_FREE -#define ATOMIC_INT_LOCK_FREE __GCC_ATOMIC_INT_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_LONG_LOCK_FREE -#define ATOMIC_LONG_LOCK_FREE __GCC_ATOMIC_LONG_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_LLONG_LOCK_FREE -#define ATOMIC_LLONG_LOCK_FREE __GCC_ATOMIC_LLONG_LOCK_FREE -#endif -#ifdef __GCC_ATOMIC_POINTER_LOCK_FREE -#define ATOMIC_POINTER_LOCK_FREE __GCC_ATOMIC_POINTER_LOCK_FREE -#endif - -/* - * 7.17.2 Initialization. - */ - -#define ATOMIC_VAR_INIT(value) (value) -#define atomic_init(obj, value) __c11_atomic_init(obj, value) - -/* - * Clang and recent GCC both provide predefined macros for the memory - * orderings. If we are using a compiler that doesn't define them, use the - * clang values - these will be ignored in the fallback path. - */ - -#ifndef __ATOMIC_RELAXED -#define __ATOMIC_RELAXED 0 -#endif -#ifndef __ATOMIC_CONSUME -#define __ATOMIC_CONSUME 1 -#endif -#ifndef __ATOMIC_ACQUIRE -#define __ATOMIC_ACQUIRE 2 -#endif -#ifndef __ATOMIC_RELEASE -#define __ATOMIC_RELEASE 3 -#endif -#ifndef __ATOMIC_ACQ_REL -#define __ATOMIC_ACQ_REL 4 -#endif -#ifndef __ATOMIC_SEQ_CST -#define __ATOMIC_SEQ_CST 5 -#endif - -/* - * 7.17.3 Order and consistency. - * - * The memory_order_* constants that denote the barrier behaviour of the - * atomic operations. - * The enum values must be identical to those used by the - * C++ <atomic> header. - */ - -typedef enum { - memory_order_relaxed = __ATOMIC_RELAXED, - memory_order_consume = __ATOMIC_CONSUME, - memory_order_acquire = __ATOMIC_ACQUIRE, - memory_order_release = __ATOMIC_RELEASE, - memory_order_acq_rel = __ATOMIC_ACQ_REL, - memory_order_seq_cst = __ATOMIC_SEQ_CST -} memory_order; - -/* - * 7.17.4 Fences. - */ - -static __inline void atomic_thread_fence(memory_order __order __attribute__((unused))) { - __c11_atomic_thread_fence(__order); -} - -static __inline void atomic_signal_fence(memory_order __order __attribute__((unused))) { - __c11_atomic_signal_fence(__order); -} - -/* - * 7.17.5 Lock-free property. - */ - -#define atomic_is_lock_free(obj) __c11_atomic_is_lock_free(sizeof(*(obj))) - -/* - * 7.17.6 Atomic integer types. - */ - -typedef _Atomic(bool) atomic_bool; -typedef _Atomic(char) atomic_char; -typedef _Atomic(signed char) atomic_schar; -typedef _Atomic(unsigned char) atomic_uchar; -typedef _Atomic(short) atomic_short; -typedef _Atomic(unsigned short) atomic_ushort; -typedef _Atomic(int) atomic_int; -typedef _Atomic(unsigned int) atomic_uint; -typedef _Atomic(long) atomic_long; -typedef _Atomic(unsigned long) atomic_ulong; -typedef _Atomic(long long) atomic_llong; -typedef _Atomic(unsigned long long) atomic_ullong; -#if defined(__BIONIC__) || (defined(__cplusplus) && __cplusplus >= 201103L) - typedef _Atomic(char16_t) atomic_char16_t; - typedef _Atomic(char32_t) atomic_char32_t; -#endif -typedef _Atomic(wchar_t) atomic_wchar_t; -typedef _Atomic(int_least8_t) atomic_int_least8_t; -typedef _Atomic(uint_least8_t) atomic_uint_least8_t; -typedef _Atomic(int_least16_t) atomic_int_least16_t; -typedef _Atomic(uint_least16_t) atomic_uint_least16_t; -typedef _Atomic(int_least32_t) atomic_int_least32_t; -typedef _Atomic(uint_least32_t) atomic_uint_least32_t; -typedef _Atomic(int_least64_t) atomic_int_least64_t; -typedef _Atomic(uint_least64_t) atomic_uint_least64_t; -typedef _Atomic(int_fast8_t) atomic_int_fast8_t; -typedef _Atomic(uint_fast8_t) atomic_uint_fast8_t; -typedef _Atomic(int_fast16_t) atomic_int_fast16_t; -typedef _Atomic(uint_fast16_t) atomic_uint_fast16_t; -typedef _Atomic(int_fast32_t) atomic_int_fast32_t; -typedef _Atomic(uint_fast32_t) atomic_uint_fast32_t; -typedef _Atomic(int_fast64_t) atomic_int_fast64_t; -typedef _Atomic(uint_fast64_t) atomic_uint_fast64_t; -typedef _Atomic(intptr_t) atomic_intptr_t; -typedef _Atomic(uintptr_t) atomic_uintptr_t; -typedef _Atomic(size_t) atomic_size_t; -typedef _Atomic(ptrdiff_t) atomic_ptrdiff_t; -typedef _Atomic(intmax_t) atomic_intmax_t; -typedef _Atomic(uintmax_t) atomic_uintmax_t; - -/* - * 7.17.7 Operations on atomic types. - */ - -/* - * Compiler-specific operations. - */ - -#define atomic_compare_exchange_strong_explicit(object, expected, \ - desired, success, failure) \ - __c11_atomic_compare_exchange_strong(object, expected, desired, \ - success, failure) -#define atomic_compare_exchange_weak_explicit(object, expected, \ - desired, success, failure) \ - __c11_atomic_compare_exchange_weak(object, expected, desired, \ - success, failure) -#define atomic_exchange_explicit(object, desired, order) \ - __c11_atomic_exchange(object, desired, order) -#define atomic_fetch_add_explicit(object, operand, order) \ - __c11_atomic_fetch_add(object, operand, order) -#define atomic_fetch_and_explicit(object, operand, order) \ - __c11_atomic_fetch_and(object, operand, order) -#define atomic_fetch_or_explicit(object, operand, order) \ - __c11_atomic_fetch_or(object, operand, order) -#define atomic_fetch_sub_explicit(object, operand, order) \ - __c11_atomic_fetch_sub(object, operand, order) -#define atomic_fetch_xor_explicit(object, operand, order) \ - __c11_atomic_fetch_xor(object, operand, order) -#define atomic_load_explicit(object, order) \ - __c11_atomic_load(object, order) -#define atomic_store_explicit(object, desired, order) \ - __c11_atomic_store(object, desired, order) - -/* - * Convenience functions. - */ - -#define atomic_compare_exchange_strong(object, expected, desired) \ - atomic_compare_exchange_strong_explicit(object, expected, \ - desired, memory_order_seq_cst, memory_order_seq_cst) -#define atomic_compare_exchange_weak(object, expected, desired) \ - atomic_compare_exchange_weak_explicit(object, expected, \ - desired, memory_order_seq_cst, memory_order_seq_cst) -#define atomic_exchange(object, desired) \ - atomic_exchange_explicit(object, desired, memory_order_seq_cst) -#define atomic_fetch_add(object, operand) \ - atomic_fetch_add_explicit(object, operand, memory_order_seq_cst) -#define atomic_fetch_and(object, operand) \ - atomic_fetch_and_explicit(object, operand, memory_order_seq_cst) -#define atomic_fetch_or(object, operand) \ - atomic_fetch_or_explicit(object, operand, memory_order_seq_cst) -#define atomic_fetch_sub(object, operand) \ - atomic_fetch_sub_explicit(object, operand, memory_order_seq_cst) -#define atomic_fetch_xor(object, operand) \ - atomic_fetch_xor_explicit(object, operand, memory_order_seq_cst) -#define atomic_load(object) \ - atomic_load_explicit(object, memory_order_seq_cst) -#define atomic_store(object, desired) \ - atomic_store_explicit(object, desired, memory_order_seq_cst) - -/* - * 7.17.8 Atomic flag type and operations. - * - * XXX: Assume atomic_bool can be used as an atomic_flag. Is there some - * kind of compiler built-in type we could use? - */ - -typedef struct { - atomic_bool __flag; -} atomic_flag; - -#define ATOMIC_FLAG_INIT { ATOMIC_VAR_INIT(false) } - -static __inline bool atomic_flag_test_and_set_explicit(volatile atomic_flag *__object, memory_order __order) { - return (atomic_exchange_explicit(&__object->__flag, 1, __order)); -} - -static __inline void atomic_flag_clear_explicit(volatile atomic_flag *__object, memory_order __order) { - atomic_store_explicit(&__object->__flag, 0, __order); -} - -static __inline bool atomic_flag_test_and_set(volatile atomic_flag *__object) { - return (atomic_flag_test_and_set_explicit(__object, memory_order_seq_cst)); -} - -static __inline void atomic_flag_clear(volatile atomic_flag *__object) { - atomic_flag_clear_explicit(__object, memory_order_seq_cst); -} +/* Actual implementation is in bits/stdatomic.h since our test code is C++. */ +#include <bits/stdatomic.h> #endif /* <atomic> unavailable */ diff --git a/libc/include/stdio.h b/libc/include/stdio.h index 8bd690fec..6632c0170 100644 --- a/libc/include/stdio.h +++ b/libc/include/stdio.h @@ -165,9 +165,53 @@ char* tmpnam(char* __s) char* tempnam(const char* __dir, const char* __prefix) __warnattr("tempnam is unsafe, use mkstemp or tmpfile instead"); +/** + * [rename(2)](http://man7.org/linux/man-pages/man2/rename.2.html) changes + * the name or location of a file. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int rename(const char* __old_path, const char* __new_path); + +/** + * [renameat(2)](http://man7.org/linux/man-pages/man2/renameat.2.html) changes + * the name or location of a file, interpreting relative paths using an fd. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int renameat(int __old_dir_fd, const char* __old_path, int __new_dir_fd, const char* __new_path); +#if defined(__USE_GNU) + +/** + * Flag for [renameat2(2)](http://man7.org/linux/man-pages/man2/renameat2.2.html) + * to fail if the new path already exists. + */ +#define RENAME_NOREPLACE (1<<0) + +/** + * Flag for [renameat2(2)](http://man7.org/linux/man-pages/man2/renameat2.2.html) + * to atomically exchange the two paths. + */ +#define RENAME_EXCHANGE (1<<1) + +/** + * Flag for [renameat2(2)](http://man7.org/linux/man-pages/man2/renameat2.2.html) + * to create a union/overlay filesystem object. + */ +#define RENAME_WHITEOUT (1<<2) + +/** + * [renameat2(2)](http://man7.org/linux/man-pages/man2/renameat2.2.html) changes + * the name or location of a file, interpreting relative paths using an fd, + * with optional `RENAME_` flags. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ +int renameat2(int __old_dir_fd, const char* __old_path, int __new_dir_fd, const char* __new_path, unsigned __flags) __INTRODUCED_IN(30); + +#endif + int fseek(FILE* __fp, long __offset, int __whence); long ftell(FILE* __fp); diff --git a/libc/include/sys/mman.h b/libc/include/sys/mman.h index 89d6d07c3..3b83229d7 100644 --- a/libc/include/sys/mman.h +++ b/libc/include/sys/mman.h @@ -26,34 +26,27 @@ * SUCH DAMAGE. */ -#ifndef _SYS_MMAN_H_ -#define _SYS_MMAN_H_ +#pragma once #include <sys/cdefs.h> #include <sys/types.h> +#include <linux/memfd.h> #include <linux/mman.h> __BEGIN_DECLS -#ifndef MAP_ANON -#define MAP_ANON MAP_ANONYMOUS -#endif +/** Alternative spelling of the `MAP_ANONYMOUS` flag for mmap(). */ +#define MAP_ANON MAP_ANONYMOUS +/** Return value for mmap(). */ #define MAP_FAILED __BIONIC_CAST(reinterpret_cast, void*, -1) -#define MREMAP_MAYMOVE 1 -#define MREMAP_FIXED 2 - -/* - * See https://android.googlesource.com/platform/bionic/+/master/docs/32-bit-abi.md - * - * mmap64 wasn't really around until L, but we added an inline for it since it - * allows a lot more code to compile with _FILE_OFFSET_BITS=64. +/** + * [mmap(2)](http://man7.org/linux/man-pages/man2/mmap.2.html) + * creates a memory mapping for the given range. * - * GCC removes the static inline unless it is explicitly used. We can get around - * this with __attribute__((used)), but that needlessly adds a definition of - * mmap64 to every translation unit that includes this header. Instead, just - * preserve the old behavior for GCC and emit a useful diagnostic. + * Returns the address of the mapping on success, + * and returns `MAP_FAILED` and sets `errno` on failure. */ #if defined(__USE_FILE_OFFSET64) void* mmap(void* __addr, size_t __size, int __prot, int __flags, int __fd, off_t __offset) __RENAME(mmap64); @@ -62,25 +55,126 @@ void* mmap(void* __addr, size_t __size, int __prot, int __flags, int __fd, off_t #endif #if __ANDROID_API__ >= __ANDROID_API_L__ +/** + * mmap64() is a variant of mmap() that takes a 64-bit offset even on LP32. + * + * See https://android.googlesource.com/platform/bionic/+/master/docs/32-bit-abi.md + * + * mmap64 wasn't really around until L, but we added an inline for it since it + * allows a lot more code to compile with _FILE_OFFSET_BITS=64. + */ void* mmap64(void* __addr, size_t __size, int __prot, int __flags, int __fd, off64_t __offset) __INTRODUCED_IN(21); #endif +/** + * [munmap(2)](http://man7.org/linux/man-pages/man2/munmap.2.html) + * deletes a memory mapping for the given range. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int munmap(void* __addr, size_t __size); + +/** + * [msync(2)](http://man7.org/linux/man-pages/man2/msync.2.html) + * flushes changes to a memory-mapped file to disk. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int msync(void* __addr, size_t __size, int __flags); + +/** + * [mprotect(2)](http://man7.org/linux/man-pages/man2/mprotect.2.html) + * sets the protection on a memory region. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int mprotect(void* __addr, size_t __size, int __prot); + +/** Flag for mremap(). */ +#define MREMAP_MAYMOVE 1 + +/** Flag for mremap(). */ +#define MREMAP_FIXED 2 + +/** + * [mremap(2)](http://man7.org/linux/man-pages/man2/mremap.2.html) + * expands or shrinks an existing memory mapping. + * + * Returns the address of the mapping on success, + * and returns `MAP_FAILED` and sets `errno` on failure. + */ void* mremap(void* __old_addr, size_t __old_size, size_t __new_size, int __flags, ...); +/** + * [mlockall(2)](http://man7.org/linux/man-pages/man2/mlockall.2.html) + * locks pages (preventing swapping). + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int mlockall(int __flags) __INTRODUCED_IN(17); + +/** + * [munlockall(2)](http://man7.org/linux/man-pages/man2/munlockall.2.html) + * unlocks pages (allowing swapping). + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int munlockall(void) __INTRODUCED_IN(17); +/** + * [mlock(2)](http://man7.org/linux/man-pages/man2/mlock.2.html) + * locks pages (preventing swapping). + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int mlock(const void* __addr, size_t __size); + +/** + * [mlock2(2)](http://man7.org/linux/man-pages/man2/mlock.2.html) + * locks pages (preventing swapping), with optional flags. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ +int mlock2(const void* __addr, size_t __size, int __flags) __INTRODUCED_IN(30); + +/** + * [munlock(2)](http://man7.org/linux/man-pages/man2/munlock.2.html) + * unlocks pages (allowing swapping). + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int munlock(const void* __addr, size_t __size); +/** + * [mincore(2)](http://man7.org/linux/man-pages/man2/mincore.2.html) + * tests whether pages are resident in memory. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int mincore(void* __addr, size_t __size, unsigned char* __vector); +/** + * [madvise(2)](http://man7.org/linux/man-pages/man2/madvise.2.html) + * gives the kernel advice about future usage patterns. + * + * Returns 0 on success, and returns -1 and sets `errno` on failure. + */ int madvise(void* __addr, size_t __size, int __advice); +#if defined(__USE_GNU) + +/** + * [memfd_create(2)](http://man7.org/linux/man-pages/man2/memfd_create.2.html) + * creates an anonymous file. + * + * Returns an fd on success, and returns -1 and sets `errno` on failure. + */ +int memfd_create(const char* __name, unsigned __flags) __INTRODUCED_IN(30); + +#endif + #if __ANDROID_API__ >= __ANDROID_API_M__ + /* * Some third-party code uses the existence of POSIX_MADV_NORMAL to detect the * availability of posix_madvise. This is not correct, since having up-to-date @@ -89,16 +183,30 @@ int madvise(void* __addr, size_t __size, int __advice); * * https://github.com/android-ndk/ndk/issues/395 */ + +/** Flag for posix_madvise(). */ #define POSIX_MADV_NORMAL MADV_NORMAL +/** Flag for posix_madvise(). */ #define POSIX_MADV_RANDOM MADV_RANDOM +/** Flag for posix_madvise(). */ #define POSIX_MADV_SEQUENTIAL MADV_SEQUENTIAL +/** Flag for posix_madvise(). */ #define POSIX_MADV_WILLNEED MADV_WILLNEED +/** Flag for posix_madvise(). */ #define POSIX_MADV_DONTNEED MADV_DONTNEED + #endif + +/** + * [posix_madvise(3)](http://man7.org/linux/man-pages/man3/posix_madvise.3.html) + * gives the kernel advice about future usage patterns. + * + * Returns 0 on success, and returns a positive error number on failure. + * + * See also madvise() which has been available much longer. + */ int posix_madvise(void* __addr, size_t __size, int __advice) __INTRODUCED_IN(23); __END_DECLS #include <android/legacy_sys_mman_inlines.h> - -#endif diff --git a/libc/kernel/tools/update_all.py b/libc/kernel/tools/update_all.py index 9d5b02dee..cef24dd4c 100755 --- a/libc/kernel/tools/update_all.py +++ b/libc/kernel/tools/update_all.py @@ -110,7 +110,9 @@ def GenerateGlibcSyscallsHeader(updater): content += ' #define SYS_%s %s\n' % (syscall, nr_name) content += '#endif\n' - updater.editFile('%s/include/bits/glibc-syscalls.h' % libc_root, content) + syscall_file = os.path.join(libc_root, 'include/bits/glibc-syscalls.h') + updater.readFile(syscall_file) + updater.editFile(syscall_file, content) try: diff --git a/libc/kernel/uapi/asm-x86/asm/vmx.h b/libc/kernel/uapi/asm-x86/asm/vmx.h index 0f7ede3b5..83365f8df 100644 --- a/libc/kernel/uapi/asm-x86/asm/vmx.h +++ b/libc/kernel/uapi/asm-x86/asm/vmx.h @@ -78,6 +78,5 @@ #define VMX_EXIT_REASONS { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, { EXIT_REASON_EXTERNAL_INTERRUPT, "EXTERNAL_INTERRUPT" }, { EXIT_REASON_TRIPLE_FAULT, "TRIPLE_FAULT" }, { EXIT_REASON_PENDING_INTERRUPT, "PENDING_INTERRUPT" }, { EXIT_REASON_NMI_WINDOW, "NMI_WINDOW" }, { EXIT_REASON_TASK_SWITCH, "TASK_SWITCH" }, { EXIT_REASON_CPUID, "CPUID" }, { EXIT_REASON_HLT, "HLT" }, { EXIT_REASON_INVD, "INVD" }, { EXIT_REASON_INVLPG, "INVLPG" }, { EXIT_REASON_RDPMC, "RDPMC" }, { EXIT_REASON_RDTSC, "RDTSC" }, { EXIT_REASON_VMCALL, "VMCALL" }, { EXIT_REASON_VMCLEAR, "VMCLEAR" }, { EXIT_REASON_VMLAUNCH, "VMLAUNCH" }, { EXIT_REASON_VMPTRLD, "VMPTRLD" }, { EXIT_REASON_VMPTRST, "VMPTRST" }, { EXIT_REASON_VMREAD, "VMREAD" }, { EXIT_REASON_VMRESUME, "VMRESUME" }, { EXIT_REASON_VMWRITE, "VMWRITE" }, { EXIT_REASON_VMOFF, "VMOFF" }, { EXIT_REASON_VMON, "VMON" }, { EXIT_REASON_CR_ACCESS, "CR_ACCESS" }, { EXIT_REASON_DR_ACCESS, "DR_ACCESS" }, { EXIT_REASON_IO_INSTRUCTION, "IO_INSTRUCTION" }, { EXIT_REASON_MSR_READ, "MSR_READ" }, { EXIT_REASON_MSR_WRITE, "MSR_WRITE" }, { EXIT_REASON_INVALID_STATE, "INVALID_STATE" }, { EXIT_REASON_MSR_LOAD_FAIL, "MSR_LOAD_FAIL" }, { EXIT_REASON_MWAIT_INSTRUCTION, "MWAIT_INSTRUCTION" }, { EXIT_REASON_MONITOR_TRAP_FLAG, "MONITOR_TRAP_FLAG" }, { EXIT_REASON_MONITOR_INSTRUCTION, "MONITOR_INSTRUCTION" }, { EXIT_REASON_PAUSE_INSTRUCTION, "PAUSE_INSTRUCTION" }, { EXIT_REASON_MCE_DURING_VMENTRY, "MCE_DURING_VMENTRY" }, { EXIT_REASON_TPR_BELOW_THRESHOLD, "TPR_BELOW_THRESHOLD" }, { EXIT_REASON_APIC_ACCESS, "APIC_ACCESS" }, { EXIT_REASON_EOI_INDUCED, "EOI_INDUCED" }, { EXIT_REASON_GDTR_IDTR, "GDTR_IDTR" }, { EXIT_REASON_LDTR_TR, "LDTR_TR" }, { EXIT_REASON_EPT_VIOLATION, "EPT_VIOLATION" }, { EXIT_REASON_EPT_MISCONFIG, "EPT_MISCONFIG" }, { EXIT_REASON_INVEPT, "INVEPT" }, { EXIT_REASON_RDTSCP, "RDTSCP" }, { EXIT_REASON_PREEMPTION_TIMER, "PREEMPTION_TIMER" }, { EXIT_REASON_INVVPID, "INVVPID" }, { EXIT_REASON_WBINVD, "WBINVD" }, { EXIT_REASON_XSETBV, "XSETBV" }, { EXIT_REASON_APIC_WRITE, "APIC_WRITE" }, { EXIT_REASON_RDRAND, "RDRAND" }, { EXIT_REASON_INVPCID, "INVPCID" }, { EXIT_REASON_VMFUNC, "VMFUNC" }, { EXIT_REASON_ENCLS, "ENCLS" }, { EXIT_REASON_RDSEED, "RDSEED" }, { EXIT_REASON_PML_FULL, "PML_FULL" }, { EXIT_REASON_XSAVES, "XSAVES" }, { EXIT_REASON_XRSTORS, "XRSTORS" } #define VMX_ABORT_SAVE_GUEST_MSR_FAIL 1 #define VMX_ABORT_LOAD_HOST_PDPTE_FAIL 2 -#define VMX_ABORT_VMCS_CORRUPTED 3 #define VMX_ABORT_LOAD_HOST_MSR_FAIL 4 #endif diff --git a/libc/kernel/uapi/linux/bpf.h b/libc/kernel/uapi/linux/bpf.h index 11df9942b..3f0942167 100644 --- a/libc/kernel/uapi/linux/bpf.h +++ b/libc/kernel/uapi/linux/bpf.h @@ -548,6 +548,7 @@ struct bpf_prog_info { char name[BPF_OBJ_NAME_LEN]; __u32 ifindex; __u32 gpl_compatible : 1; + __u32 : 31; __u64 netns_dev; __u64 netns_ino; __u32 nr_jited_ksyms; diff --git a/libc/kernel/uapi/linux/nilfs2_ondisk.h b/libc/kernel/uapi/linux/nilfs2_ondisk.h index e9995a11a..a05e7b732 100644 --- a/libc/kernel/uapi/linux/nilfs2_ondisk.h +++ b/libc/kernel/uapi/linux/nilfs2_ondisk.h @@ -20,6 +20,7 @@ #define _LINUX_NILFS2_ONDISK_H #include <linux/types.h> #include <linux/magic.h> +#include <asm/byteorder.h> #define NILFS_INODE_BMAP_SIZE 7 struct nilfs_inode { __le64 i_blocks; @@ -248,11 +249,11 @@ enum { NILFS_CHECKPOINT_MINOR, }; #define NILFS_CHECKPOINT_FNS(flag,name) static inline void nilfs_checkpoint_set_ ##name(struct nilfs_checkpoint * cp) \ -{ cp->cp_flags = cpu_to_le32(le32_to_cpu(cp->cp_flags) | (1UL << NILFS_CHECKPOINT_ ##flag)); \ +{ cp->cp_flags = __cpu_to_le32(__le32_to_cpu(cp->cp_flags) | (1UL << NILFS_CHECKPOINT_ ##flag)); \ } static inline void nilfs_checkpoint_clear_ ##name(struct nilfs_checkpoint * cp) \ -{ cp->cp_flags = cpu_to_le32(le32_to_cpu(cp->cp_flags) & ~(1UL << NILFS_CHECKPOINT_ ##flag)); \ +{ cp->cp_flags = __cpu_to_le32(__le32_to_cpu(cp->cp_flags) & ~(1UL << NILFS_CHECKPOINT_ ##flag)); \ } static inline int nilfs_checkpoint_ ##name(const struct nilfs_checkpoint * cp) \ -{ return ! ! (le32_to_cpu(cp->cp_flags) & (1UL << NILFS_CHECKPOINT_ ##flag)); \ +{ return ! ! (__le32_to_cpu(cp->cp_flags) & (1UL << NILFS_CHECKPOINT_ ##flag)); \ } struct nilfs_cpfile_header { __le64 ch_ncheckpoints; @@ -272,11 +273,11 @@ enum { NILFS_SEGMENT_USAGE_ERROR, }; #define NILFS_SEGMENT_USAGE_FNS(flag,name) static inline void nilfs_segment_usage_set_ ##name(struct nilfs_segment_usage * su) \ -{ su->su_flags = cpu_to_le32(le32_to_cpu(su->su_flags) | (1UL << NILFS_SEGMENT_USAGE_ ##flag)); \ +{ su->su_flags = __cpu_to_le32(__le32_to_cpu(su->su_flags) | (1UL << NILFS_SEGMENT_USAGE_ ##flag)); \ } static inline void nilfs_segment_usage_clear_ ##name(struct nilfs_segment_usage * su) \ -{ su->su_flags = cpu_to_le32(le32_to_cpu(su->su_flags) & ~(1UL << NILFS_SEGMENT_USAGE_ ##flag)); \ +{ su->su_flags = __cpu_to_le32(__le32_to_cpu(su->su_flags) & ~(1UL << NILFS_SEGMENT_USAGE_ ##flag)); \ } static inline int nilfs_segment_usage_ ##name(const struct nilfs_segment_usage * su) \ -{ return ! ! (le32_to_cpu(su->su_flags) & (1UL << NILFS_SEGMENT_USAGE_ ##flag)); \ +{ return ! ! (__le32_to_cpu(su->su_flags) & (1UL << NILFS_SEGMENT_USAGE_ ##flag)); \ } struct nilfs_sufile_header { __le64 sh_ncleansegs; diff --git a/libc/kernel/uapi/linux/version.h b/libc/kernel/uapi/linux/version.h index 815f53d96..94bf4ba5e 100644 --- a/libc/kernel/uapi/linux/version.h +++ b/libc/kernel/uapi/linux/version.h @@ -16,5 +16,5 @@ *** **************************************************************************** ****************************************************************************/ -#define LINUX_VERSION_CODE 328192 +#define LINUX_VERSION_CODE 328198 #define KERNEL_VERSION(a,b,c) (((a) << 16) + ((b) << 8) + (c)) diff --git a/libc/kernel/uapi/linux/videodev2.h b/libc/kernel/uapi/linux/videodev2.h index e6df45c8a..f77c6bda5 100644 --- a/libc/kernel/uapi/linux/videodev2.h +++ b/libc/kernel/uapi/linux/videodev2.h @@ -210,7 +210,7 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_RGBX444 v4l2_fourcc('R', 'X', '1', '2') #define V4L2_PIX_FMT_ABGR444 v4l2_fourcc('A', 'B', '1', '2') #define V4L2_PIX_FMT_XBGR444 v4l2_fourcc('X', 'B', '1', '2') -#define V4L2_PIX_FMT_BGRA444 v4l2_fourcc('B', 'A', '1', '2') +#define V4L2_PIX_FMT_BGRA444 v4l2_fourcc('G', 'A', '1', '2') #define V4L2_PIX_FMT_BGRX444 v4l2_fourcc('B', 'X', '1', '2') #define V4L2_PIX_FMT_RGB555 v4l2_fourcc('R', 'G', 'B', 'O') #define V4L2_PIX_FMT_ARGB555 v4l2_fourcc('A', 'R', '1', '5') diff --git a/libc/libc.map.txt b/libc/libc.map.txt index 0bcf761e1..9b39bb842 100644 --- a/libc/libc.map.txt +++ b/libc/libc.map.txt @@ -1485,6 +1485,7 @@ LIBC_R { # introduced=R __mempcpy_chk; __open64_2; __openat64_2; + __tls_get_addr; # arm64 call_once; cnd_broadcast; cnd_destroy; @@ -1492,6 +1493,8 @@ LIBC_R { # introduced=R cnd_signal; cnd_timedwait; cnd_wait; + memfd_create; + mlock2; mtx_destroy; mtx_init; mtx_lock; @@ -1502,6 +1505,7 @@ LIBC_R { # introduced=R pthread_mutex_clocklock; pthread_rwlock_clockrdlock; pthread_rwlock_clockwrlock; + renameat2; sem_clockwait; thrd_create; thrd_current; diff --git a/libc/malloc_debug/Android.bp b/libc/malloc_debug/Android.bp index aae16f14d..d6b85319e 100644 --- a/libc/malloc_debug/Android.bp +++ b/libc/malloc_debug/Android.bp @@ -16,7 +16,6 @@ cc_library_static { whole_static_libs: [ "libbase", "libasync_safe", - "libdemangle", ], include_dirs: ["bionic/libc"], @@ -66,7 +65,6 @@ cc_library { static_libs: [ "libasync_safe", "libbase", - "libdemangle", "libc_malloc_debug_backtrace", ], @@ -122,7 +120,6 @@ cc_test { static_libs: [ "libc_malloc_debug", - "libdemangle", "libtinyxml2", ], diff --git a/libc/malloc_debug/PointerData.cpp b/libc/malloc_debug/PointerData.cpp index 617d128c7..ec7e42dda 100644 --- a/libc/malloc_debug/PointerData.cpp +++ b/libc/malloc_debug/PointerData.cpp @@ -43,7 +43,6 @@ #include <android-base/stringprintf.h> #include <android-base/thread_annotations.h> -#include <demangle.h> #include <private/bionic_macros.h> #include "Config.h" @@ -54,6 +53,8 @@ #include "malloc_debug.h" #include "UnwindBacktrace.h" +extern "C" char* __cxa_demangle(const char*, char*, size_t*, int*); + std::atomic_uint8_t PointerData::backtrace_enabled_; std::atomic_bool PointerData::backtrace_dump_; @@ -596,7 +597,16 @@ void PointerData::DumpLiveToFile(FILE* fp) { if (frame.function_name.empty()) { fprintf(fp, " \"\" 0}"); } else { - fprintf(fp, " \"%s\" %" PRIx64 "}", demangle(frame.function_name.c_str()).c_str(), frame.function_offset); + char* demangled_name = __cxa_demangle(frame.function_name.c_str(), nullptr, nullptr, + nullptr); + const char* name; + if (demangled_name != nullptr) { + name = demangled_name; + } else { + name = frame.function_name.c_str(); + } + fprintf(fp, " \"%s\" %" PRIx64 "}", name, frame.function_offset); + free(demangled_name); } } fprintf(fp, "\n"); diff --git a/libc/malloc_debug/UnwindBacktrace.cpp b/libc/malloc_debug/UnwindBacktrace.cpp index ddafc731f..92fb3fa96 100644 --- a/libc/malloc_debug/UnwindBacktrace.cpp +++ b/libc/malloc_debug/UnwindBacktrace.cpp @@ -36,7 +36,6 @@ #include <vector> #include <android-base/stringprintf.h> -#include <demangle.h> #include <unwindstack/LocalUnwinder.h> #include <unwindstack/MapInfo.h> @@ -49,6 +48,8 @@ #define PAD_PTR "08" PRIx64 #endif +extern "C" char* __cxa_demangle(const char*, char*, size_t*, int*); + static pthread_once_t g_setup_once = PTHREAD_ONCE_INIT; static unwindstack::LocalUnwinder* g_unwinder; @@ -100,7 +101,14 @@ void UnwindLog(const std::vector<unwindstack::LocalFrameData>& frame_info) { } if (!info->function_name.empty()) { - line += " (" + demangle(info->function_name.c_str()); + line += " ("; + char* demangled_name = __cxa_demangle(info->function_name.c_str(), nullptr, nullptr, nullptr); + if (demangled_name != nullptr) { + line += demangled_name; + free(demangled_name); + } else { + line += info->function_name; + } if (info->function_offset != 0) { line += "+" + std::to_string(info->function_offset); } diff --git a/libc/malloc_debug/backtrace.cpp b/libc/malloc_debug/backtrace.cpp index 0e3a53f92..ab5c50523 100644 --- a/libc/malloc_debug/backtrace.cpp +++ b/libc/malloc_debug/backtrace.cpp @@ -36,8 +36,6 @@ #include <unistd.h> #include <unwind.h> -#include <demangle.h> - #include "MapData.h" #include "backtrace.h" #include "debug_log.h" @@ -164,10 +162,18 @@ std::string backtrace_string(const uintptr_t* frames, size_t frame_count) { char buf[1024]; if (symbol != nullptr) { + char* demangled_name = __cxa_demangle(symbol, nullptr, nullptr, nullptr); + const char* name; + if (demangled_name != nullptr) { + name = demangled_name; + } else { + name = symbol; + } async_safe_format_buffer(buf, sizeof(buf), " #%02zd pc %" PAD_PTR " %s%s (%s+%" PRIuPTR ")\n", - frame_num, rel_pc, soname, offset_buf, demangle(symbol).c_str(), + frame_num, rel_pc, soname, offset_buf, name, frames[frame_num] - offset); + free(demangled_name); } else { async_safe_format_buffer(buf, sizeof(buf), " #%02zd pc %" PAD_PTR " %s%s\n", frame_num, rel_pc, soname, offset_buf); diff --git a/libc/private/bionic_globals.h b/libc/private/bionic_globals.h index d73079e33..ef735bab0 100644 --- a/libc/private/bionic_globals.h +++ b/libc/private/bionic_globals.h @@ -88,6 +88,10 @@ struct libc_shared_globals { TlsModules tls_modules; BionicAllocator tls_allocator; + // Values passed from the HWASan runtime (via libc.so) to the loader. + void (*load_hook)(ElfW(Addr) base, const ElfW(Phdr)* phdr, ElfW(Half) phnum) = nullptr; + void (*unload_hook)(ElfW(Addr) base, const ElfW(Phdr)* phdr, ElfW(Half) phnum) = nullptr; + // Values passed from the linker to libc.so. const char* init_progname = nullptr; char** init_environ = nullptr; diff --git a/libc/private/sigrtmin.h b/libc/private/sigrtmin.h index d78d98058..5de1a32ae 100644 --- a/libc/private/sigrtmin.h +++ b/libc/private/sigrtmin.h @@ -39,13 +39,14 @@ // 33 (__SIGRTMIN + 1) libbacktrace // 34 (__SIGRTMIN + 2) libcore // 35 (__SIGRTMIN + 3) debuggerd -b -// 36 (__SIGRTMIN + 4) heapprofd +// 36 (__SIGRTMIN + 4) heapprofd native dumps // 37 (__SIGRTMIN + 5) coverage (libprofile-extras) +// 38 (__SIGRTMIN + 6) heapprofd ART managed heap dumps // // If you change this, also change __ndk_legacy___libc_current_sigrtmin // in <android/legacy_signal_inlines.h> to match. -#define __SIGRT_RESERVED 6 +#define __SIGRT_RESERVED 7 static inline __always_inline sigset64_t filter_reserved_signals(sigset64_t sigset, int how) { int (*block)(sigset64_t*, int); int (*unblock)(sigset64_t*, int); @@ -72,5 +73,6 @@ static inline __always_inline sigset64_t filter_reserved_signals(sigset64_t sigs unblock(&sigset, __SIGRTMIN + 3); unblock(&sigset, __SIGRTMIN + 4); unblock(&sigset, __SIGRTMIN + 5); + unblock(&sigset, __SIGRTMIN + 6); return sigset; } diff --git a/libc/symbol_ordering b/libc/symbol_ordering deleted file mode 100644 index c04692bca..000000000 --- a/libc/symbol_ordering +++ /dev/null @@ -1,210 +0,0 @@ -# This file is generated by sorting symbols in the .bss section in libc.so by -# their sizes and taking out symbols that are unique to a target. By sorting -# symbols by size, we usually have less dirty pages at runtime, because small -# symbols are grouped together. - -ctl_initialized -gGlobalsMutating -global_hashtable_initialized -gmtcheck.gmt_is_set -had_conf_error -je_background_thread_enabled_state -je_log_init_done -je_opt_abort -je_opt_abort_conf -je_opt_background_thread -je_opt_junk_alloc -je_opt_junk_free -je_opt_stats_print -je_opt_utrace -je_opt_xmalloc -je_opt_zero -je_tsd_booted -malloc_disabled_tcache -malloc_slow_flags -mmap_flags -os_overcommits -restartloop -_ZL24gHeapprofdInitInProgress -_ZL27gHeapprofdInitHookInstalled -_ZL23gZygoteChildProfileable -_ZZ17__find_icu_symbolPKcE9found_icu -ru_a -ru_b -ru_counter -ru_g -ru_msb -ru_seed -ru_seed2 -ru_x -_ZZ12bindresvportE4port -a0 -__atexit -atexit_mutex -b0 -ctl_arenas -ctl_stats -__cxa_finalize.call_depth -daylight -dirty_decay_ms_default.0 -environ -error_message_count -error_one_per_line -error_print_progname -__free_hook -g_atexit_lock -gGlobalsMutateLock -global_hashtable -gmtptr -handlers -je_background_thread_info -je_hooks_arena_new_hook -je_init_system_thp_mode -je_malloc_conf -je_malloc_disable.once_control -je_malloc_message -je_narenas_auto -je_ncpus -je_nhbins -je_opt_dirty_decay_ms -je_opt_metadata_thp -je_opt_muzzy_decay_ms -je_opt_narenas -je_opt_thp -je_tcache_bin_info -je_tcache_maxclass -je_tcaches -je_tsd_tsd -lastenv -lcl_is_set -lclptr -locallock -malloc_disabled_lock -__malloc_hook -malloc_initializer -mbrlen.mbs -mbtowc.mbs -__memalign_hook -muzzy_decay_ms_default.0 -narenas_total -ncleanups -optarg -optreset -os_page -p5s -__progname -random_mutex -__realloc_hook -_res_cache_list_lock -_res_cache_once -_res_key -__res_randomid.__libc_mutex_random -rs -_rs_forkdetect._rs_pid -_rs_forked -rsx -ru_pid -ru_prf -ru_reseed -__stack_chk_guard -stack_nelms -strtok.last -suboptarg -__system_property_area__ -tcaches_avail -tcaches_past -timezone -tmpnam.tmpcount -ut -wctomb.mbs -_ZL11g_arc4_lock -_ZL13g_locale_once -_ZL13g_thread_list -_ZL14syslog_log_tag -_ZL16gHeapprofdHandle -_ZL17g_libicuuc_handle -_ZL18netdClientInitOnce -_ZL30g_propservice_protocol_version -_ZN9prop_area13pa_data_size_E -_ZN9prop_area8pa_size_E -_ZZ10mbsnrtowcsE15__private_state -_ZZ10wcsnrtombsE15__private_state -_ZZ13error_at_lineE9last_file -_ZZ13error_at_lineE9last_line -_ZZ14__icu_charTypejE10u_charType -_ZZ23__bionic_get_shell_pathE7sh_path -_ZZ23__icu_hasBinaryPropertyj9UPropertyPFiiEE19u_hasBinaryProperty -_ZZ25__icu_getIntPropertyValuej9UPropertyE21u_getIntPropertyValue -_ZZ7mbrtowcE15__private_state -_ZZ7wcrtombE15__private_state -_ZZ8c16rtombE15__private_state -_ZZ8c32rtombE15__private_state -_ZZ8iswcntrlE10u_charType -_ZZ8iswdigitE9u_isdigit -_ZZ8iswpunctE9u_ispunct -_ZZ8mbrtoc16E15__private_state -_ZZ8mbrtoc32E15__private_state -_ZZ8towlowerE9u_tolower -_ZZ8towupperE9u_toupper -ether_aton.addr -seed48.sseed -__dtoa_locks -_ZGVZ14__icu_charTypejE10u_charType -_ZGVZ14tzset_unlockedE20persist_sys_timezone -_ZGVZ17__find_icu_symbolPKcE9found_icu -_ZGVZ23__bionic_get_shell_pathE7sh_path -_ZGVZ23__icu_hasBinaryPropertyj9UPropertyPFiiEE19u_hasBinaryProperty -_ZGVZ25__icu_getIntPropertyValuej9UPropertyE21u_getIntPropertyValue -_ZGVZ8iswcntrlE10u_charType -_ZGVZ8iswdigitE9u_isdigit -_ZGVZ8iswpunctE9u_ispunct -_ZGVZ8towlowerE9u_tolower -_ZGVZ8towupperE9u_toupper -_ZL10gAllocated -_ZL11gAllocLimit -_ZL13g_atfork_list -_ZL6g_lock -_ZL6g_tags -je_opt_stats_print_opts -nuls -precsize_ntoa.retbuf -__p_secstodate.output -ether_ntoa.buf -inet_ntoa.b -__p_class.classbuf -__p_type.typebuf -__sym_ntop.unname -__sym_ntos.unname -_ZL10gFunctions -_ZL12vendor_group -_ZL13vendor_passwd -freelist -__p_option.nbuf -__p_time.nbuf -_ZL18g_thread_list_lock -tm -_ZL8g_locale -ctl_mtx -init_lock -je_arenas_lock -je_background_thread_lock -tcaches_mtx -je_tsd_init_head -buf_asctime -__loc_ntoa.tmpbuf -utmp -_ZZ14tzset_unlockedE20persist_sys_timezone -arena_binind_div_info -__hexdig_D2A -lcl_TZname -inet_nsap_ntoa_tmpbuf -_ZL7key_map -_ZL17system_properties -private_mem -_res_cache_list -__libc_globals -tmpnam.buf -je_extents_rtree -_nres -je_arenas -je_extent_mutex_pool diff --git a/libc/versioner-dependencies/common/clang-builtins b/libc/versioner-dependencies/common/clang-builtins index 7bd481c94..148dd2b63 120000 --- a/libc/versioner-dependencies/common/clang-builtins +++ b/libc/versioner-dependencies/common/clang-builtins @@ -1 +1 @@ -../../../../external/clang/lib/Headers/
\ No newline at end of file +../../../../prebuilts/clang-tools/linux-x86/clang-headers
\ No newline at end of file diff --git a/linker/Android.bp b/linker/Android.bp index 5e7a921a4..bdb7c178a 100644 --- a/linker/Android.bp +++ b/linker/Android.bp @@ -177,11 +177,6 @@ cc_defaults { "-Wextra", "-Wunused", "-Werror", - - // Define _USING_LIBCXX so <stdatomic.h> defers to the <atomic> header. When a Soong module - // uses the platform libc++, Soong automatically passes this macro, but the dynamic linker - // links against libc++ manually. - "-D_USING_LIBCXX", ], // TODO: split out the asflags. diff --git a/linker/ld.config.format.md b/linker/ld.config.format.md index faf5cc8a6..f9fbcde3f 100644 --- a/linker/ld.config.format.md +++ b/linker/ld.config.format.md @@ -25,7 +25,7 @@ configuration using set of properties described in example below. ## Example ``` -# The following line maps section to a dir. Binraies ran from this location will use namespaces +# The following line maps section to a dir. Binaries ran from this location will use namespaces # configuration specified in [example_section] below dir.example_section=/system/bin/example @@ -38,7 +38,7 @@ dir.example_section=/system/bin/example # default value is false enable.target.sdk.version = true -# This property can be used to declare additional namespaces.Note that there is always the default +# This property can be used to declare additional namespaces. Note that there is always the default # namespace. The default namespace is the namespace for the main executable. This list is # comma-separated. additional.namespaces = ns1 @@ -65,7 +65,7 @@ namespace.default.asan.permitted.paths = /data/${LIB} # This declares linked namespaces - comma separated list. namespace.default.links = ns1 -# For every link define list of shared libraries. This is list of the libraries accessilbe from +# For every link define list of shared libraries. This is list of the libraries accessible from # default namespace but loaded in the linked namespace. namespace.default.link.ns1.shared_libs = libexternal.so:libother.so diff --git a/linker/linker.cpp b/linker/linker.cpp index 55d684fa7..0361a8ab3 100644 --- a/linker/linker.cpp +++ b/linker/linker.cpp @@ -282,6 +282,16 @@ static linker_stats_t linker_stats; void count_relocation(RelocationKind kind) { ++linker_stats.count[kind]; } + +void print_linker_stats() { + PRINT("RELO STATS: %s: %d abs, %d rel, %d copy, %d symbol (%d cached)", + g_argv[0], + linker_stats.count[kRelocAbsolute], + linker_stats.count[kRelocRelative], + linker_stats.count[kRelocCopy], + linker_stats.count[kRelocSymbol], + linker_stats.count[kRelocSymbolCached]); +} #else void count_relocation(RelocationKind) { } @@ -1895,6 +1905,9 @@ bool find_libraries(android_namespace_t* ns, !get_cfi_shadow()->AfterLoad(si, solist_get_head())) { return false; } + if (__libc_shared_globals()->load_hook) { + __libc_shared_globals()->load_hook(si->load_bias, si->phdr, si->phnum); + } } return true; @@ -2029,6 +2042,9 @@ static void soinfo_unload_impl(soinfo* root) { si); notify_gdb_of_unload(si); unregister_soinfo_tls(si); + if (__libc_shared_globals()->unload_hook) { + __libc_shared_globals()->unload_hook(si->load_bias, si->phdr, si->phnum); + } get_cfi_shadow()->BeforeUnload(si); soinfo_free(si); } @@ -2883,6 +2899,17 @@ bool soinfo::relocate(const VersionTracker& version_tracker, ElfRelIteratorT&& r const size_t tls_tp_base = __libc_shared_globals()->static_tls_layout.offset_thread_pointer(); std::vector<std::pair<TlsDescriptor*, size_t>> deferred_tlsdesc_relocs; + struct { + // Cache key + ElfW(Word) sym; + + // Cache value + const ElfW(Sym)* s; + soinfo* lsi; + } symbol_lookup_cache; + + symbol_lookup_cache.sym = 0; + for (size_t idx = 0; rel_iterator.has_next(); ++idx) { const auto rel = rel_iterator.next(); if (rel == nullptr) { @@ -2926,14 +2953,25 @@ bool soinfo::relocate(const VersionTracker& version_tracker, ElfRelIteratorT&& r return false; } else { sym_name = get_string(symtab_[sym].st_name); - const version_info* vi = nullptr; - if (!lookup_version_info(version_tracker, sym, sym_name, &vi)) { - return false; - } + if (sym == symbol_lookup_cache.sym) { + s = symbol_lookup_cache.s; + lsi = symbol_lookup_cache.lsi; + count_relocation(kRelocSymbolCached); + } else { + const version_info* vi = nullptr; - if (!soinfo_do_lookup(this, sym_name, vi, &lsi, global_group, local_group, &s)) { - return false; + if (!lookup_version_info(version_tracker, sym, sym_name, &vi)) { + return false; + } + + if (!soinfo_do_lookup(this, sym_name, vi, &lsi, global_group, local_group, &s)) { + return false; + } + + symbol_lookup_cache.sym = sym; + symbol_lookup_cache.s = s; + symbol_lookup_cache.lsi = lsi; } if (s == nullptr) { @@ -3130,10 +3168,6 @@ bool soinfo::relocate(const VersionTracker& version_tracker, ElfRelIteratorT&& r *reinterpret_cast<ElfW(Addr)*>(reloc) = tpoff; } break; - -#if !defined(__aarch64__) - // Omit support for DTPMOD/DTPREL on arm64, at least until - // http://b/123385182 is fixed. arm64 uses TLSDESC instead. case R_GENERIC_TLS_DTPMOD: count_relocation(kRelocRelative); MARK(rel->r_offset); @@ -3158,7 +3192,6 @@ bool soinfo::relocate(const VersionTracker& version_tracker, ElfRelIteratorT&& r reinterpret_cast<void*>(sym_addr + addend), sym_name); *reinterpret_cast<ElfW(Addr)*>(reloc) = sym_addr + addend; break; -#endif // !defined(__aarch64__) #if defined(__aarch64__) // Bionic currently only implements TLSDESC for arm64. This implementation should work with diff --git a/linker/linker.h b/linker/linker.h index 0f3094616..782da1c51 100644 --- a/linker/linker.h +++ b/linker/linker.h @@ -94,11 +94,14 @@ enum RelocationKind { kRelocRelative, kRelocCopy, kRelocSymbol, + kRelocSymbolCached, kRelocMax }; void count_relocation(RelocationKind kind); +void print_linker_stats(); + soinfo* get_libdl_info(const soinfo& linker_si); soinfo* find_containing_library(const void* p); diff --git a/linker/linker_main.cpp b/linker/linker_main.cpp index 612f52f88..11d3d2914 100644 --- a/linker/linker_main.cpp +++ b/linker/linker_main.cpp @@ -490,11 +490,7 @@ static ElfW(Addr) linker_main(KernelArgumentBlock& args, const char* exe_to_load static_cast<long long>(t0.tv_usec)))); #endif #if STATS - PRINT("RELO STATS: %s: %d abs, %d rel, %d copy, %d symbol", g_argv[0], - linker_stats.count[kRelocAbsolute], - linker_stats.count[kRelocRelative], - linker_stats.count[kRelocCopy], - linker_stats.count[kRelocSymbol]); + print_linker_stats(); #endif #if COUNT_PAGES { diff --git a/linker/linker_relocs.h b/linker/linker_relocs.h index 68191f911..1da5ebe34 100644 --- a/linker/linker_relocs.h +++ b/linker/linker_relocs.h @@ -38,9 +38,9 @@ #define R_GENERIC_GLOB_DAT R_AARCH64_GLOB_DAT #define R_GENERIC_RELATIVE R_AARCH64_RELATIVE #define R_GENERIC_IRELATIVE R_AARCH64_IRELATIVE -#define R_GENERIC_TLS_DTPMOD R_AARCH64_TLS_DTPMOD64 -#define R_GENERIC_TLS_DTPREL R_AARCH64_TLS_DTPREL64 -#define R_GENERIC_TLS_TPREL R_AARCH64_TLS_TPREL64 +#define R_GENERIC_TLS_DTPMOD R_AARCH64_TLS_DTPMOD +#define R_GENERIC_TLS_DTPREL R_AARCH64_TLS_DTPREL +#define R_GENERIC_TLS_TPREL R_AARCH64_TLS_TPREL #define R_GENERIC_TLSDESC R_AARCH64_TLSDESC #elif defined (__arm__) diff --git a/tests/dlext_test.cpp b/tests/dlext_test.cpp index 67ebf37df..59cf2f771 100644 --- a/tests/dlext_test.cpp +++ b/tests/dlext_test.cpp @@ -29,9 +29,7 @@ #include <android-base/file.h> #include <android-base/strings.h> -#include <linux/memfd.h> #include <sys/mman.h> -#include <sys/syscall.h> #include <sys/types.h> #include <sys/vfs.h> #include <sys/wait.h> @@ -942,7 +940,7 @@ TEST(dlext, dlopen_ext_use_memfd) { const std::string lib_path = GetTestlibRoot() + "/libtest_simple.so"; // create memfd - int memfd = syscall(__NR_memfd_create, "foobar", MFD_CLOEXEC); + int memfd = memfd_create("foobar", MFD_CLOEXEC); if (memfd == -1 && errno == ENOSYS) { return; } diff --git a/tests/grp_pwd_test.cpp b/tests/grp_pwd_test.cpp index 5b6eed863..6b49e298e 100644 --- a/tests/grp_pwd_test.cpp +++ b/tests/grp_pwd_test.cpp @@ -75,11 +75,7 @@ static void check_passwd(const passwd* pwd, const char* username, uid_t uid, uid EXPECT_STREQ("/", pwd->pw_dir); } - if (uid_type == TYPE_VENDOR) { - EXPECT_STREQ("/vendor/bin/sh", pwd->pw_shell); - } else { - EXPECT_STREQ("/system/bin/sh", pwd->pw_shell); - } + EXPECT_STREQ("/bin/sh", pwd->pw_shell); } static void check_getpwuid(const char* username, uid_t uid, uid_type_t uid_type, @@ -382,7 +378,7 @@ TEST(pwd, getpwnam_r_large_enough_suggested_buffer_size) { #if defined(__BIONIC__) template <typename T> -static void expect_ids(const T& ids, bool is_group) { +static void expect_ids(T ids, bool is_group) { std::set<typename T::key_type> expected_ids; // Ensure that all android_ids are iterated through. for (size_t n = 0; n < android_id_count; ++n) { @@ -415,6 +411,14 @@ static void expect_ids(const T& ids, bool is_group) { return; } + auto allow_range = [&ids](uid_t start, uid_t end) { + for (size_t n = start; n <= end; ++n) { + ids.erase(n); + } + }; + + allow_range(AID_SYSTEM_RESERVED_START, AID_SYSTEM_EXT_RESERVED_END); + // Ensure that no other ids were returned. auto return_differences = [&ids, &expected_ids] { std::vector<typename T::key_type> missing_from_ids; diff --git a/tests/pthread_test.cpp b/tests/pthread_test.cpp index 3b3c585f0..d8257381c 100644 --- a/tests/pthread_test.cpp +++ b/tests/pthread_test.cpp @@ -1624,14 +1624,27 @@ TEST_F(pthread_CondWakeupTest, signal_clockwait_realtime_realtime) { #endif // __BIONIC__ } -static void pthread_cond_timedwait_timeout_helper(clockid_t clock, +static void pthread_cond_timedwait_timeout_helper(bool init_monotonic, clockid_t clock, int (*wait_function)(pthread_cond_t* __cond, pthread_mutex_t* __mutex, const timespec* __timeout)) { pthread_mutex_t mutex; ASSERT_EQ(0, pthread_mutex_init(&mutex, nullptr)); pthread_cond_t cond; - ASSERT_EQ(0, pthread_cond_init(&cond, nullptr)); + + if (init_monotonic) { + pthread_condattr_t attr; + pthread_condattr_init(&attr); + + ASSERT_EQ(0, pthread_condattr_setclock(&attr, CLOCK_MONOTONIC)); + clockid_t clock; + ASSERT_EQ(0, pthread_condattr_getclock(&attr, &clock)); + ASSERT_EQ(CLOCK_MONOTONIC, clock); + + ASSERT_EQ(0, pthread_cond_init(&cond, &attr)); + } else { + ASSERT_EQ(0, pthread_cond_init(&cond, nullptr)); + } ASSERT_EQ(0, pthread_mutex_lock(&mutex)); timespec ts; @@ -1648,12 +1661,13 @@ static void pthread_cond_timedwait_timeout_helper(clockid_t clock, } TEST(pthread, pthread_cond_timedwait_timeout) { - pthread_cond_timedwait_timeout_helper(CLOCK_REALTIME, pthread_cond_timedwait); + pthread_cond_timedwait_timeout_helper(false, CLOCK_REALTIME, pthread_cond_timedwait); } TEST(pthread, pthread_cond_timedwait_monotonic_np_timeout) { #if defined(__BIONIC__) - pthread_cond_timedwait_timeout_helper(CLOCK_MONOTONIC, pthread_cond_timedwait_monotonic_np); + pthread_cond_timedwait_timeout_helper(false, CLOCK_MONOTONIC, pthread_cond_timedwait_monotonic_np); + pthread_cond_timedwait_timeout_helper(true, CLOCK_MONOTONIC, pthread_cond_timedwait_monotonic_np); #else // __BIONIC__ GTEST_SKIP() << "pthread_cond_timedwait_monotonic_np not available"; #endif // __BIONIC__ @@ -1662,12 +1676,22 @@ TEST(pthread, pthread_cond_timedwait_monotonic_np_timeout) { TEST(pthread, pthread_cond_clockwait_timeout) { #if defined(__BIONIC__) pthread_cond_timedwait_timeout_helper( - CLOCK_MONOTONIC, + false, CLOCK_MONOTONIC, [](pthread_cond_t* __cond, pthread_mutex_t* __mutex, const timespec* __timeout) { return pthread_cond_clockwait(__cond, __mutex, CLOCK_MONOTONIC, __timeout); }); pthread_cond_timedwait_timeout_helper( - CLOCK_REALTIME, + true, CLOCK_MONOTONIC, + [](pthread_cond_t* __cond, pthread_mutex_t* __mutex, const timespec* __timeout) { + return pthread_cond_clockwait(__cond, __mutex, CLOCK_MONOTONIC, __timeout); + }); + pthread_cond_timedwait_timeout_helper( + false, CLOCK_REALTIME, + [](pthread_cond_t* __cond, pthread_mutex_t* __mutex, const timespec* __timeout) { + return pthread_cond_clockwait(__cond, __mutex, CLOCK_REALTIME, __timeout); + }); + pthread_cond_timedwait_timeout_helper( + true, CLOCK_REALTIME, [](pthread_cond_t* __cond, pthread_mutex_t* __mutex, const timespec* __timeout) { return pthread_cond_clockwait(__cond, __mutex, CLOCK_REALTIME, __timeout); }); diff --git a/tests/stdatomic_test.cpp b/tests/stdatomic_test.cpp index d122d2f39..a9665d18e 100644 --- a/tests/stdatomic_test.cpp +++ b/tests/stdatomic_test.cpp @@ -15,9 +15,14 @@ */ #include <gtest/gtest.h> -// Fool stdatomic.h into not using <atomic>. -#undef _USING_LIBCXX + +#if defined(__ANDROID__) +#include <bits/stdatomic.h> +#else +#undef _USING_LIBCXX //TODO(b/137876753): Remove this #include <stdatomic.h> +#endif + #include <pthread.h> #include <stdint.h> diff --git a/tests/stdio_test.cpp b/tests/stdio_test.cpp index 01b4dbab7..a0cda1be0 100644 --- a/tests/stdio_test.cpp +++ b/tests/stdio_test.cpp @@ -19,7 +19,6 @@ #include <errno.h> #include <fcntl.h> #include <limits.h> -#include <linux/fs.h> #include <math.h> #include <stdio.h> #include <sys/types.h> @@ -34,10 +33,18 @@ #include <vector> #include <android-base/file.h> +#include <android-base/unique_fd.h> #include "BionicDeathTest.h" #include "utils.h" +// This #include is actually a test too. We have to duplicate the +// definitions of the RENAME_ constants because <linux/fs.h> also contains +// pollution such as BLOCK_SIZE which conflicts with lots of user code. +// Important to check that we have matching definitions. +// There's no _MAX to test that we have all the constants, sadly. +#include <linux/fs.h> + #if defined(NOFORTIFY) #define STDIO_TEST stdio_nofortify #define STDIO_DEATHTEST stdio_nofortify_DeathTest @@ -2610,3 +2617,77 @@ TEST(STDIO_TEST, SEEK_macros) { // So we'll notice if Linux grows another constant in <linux/fs.h>... ASSERT_EQ(SEEK_MAX, SEEK_HOLE); } + +TEST(STDIO_TEST, rename) { + TemporaryDir td; + std::string old_path = td.path + "/old"s; + std::string new_path = td.path + "/new"s; + + // Create the file, check it exists. + ASSERT_EQ(0, close(creat(old_path.c_str(), 0666))); + struct stat sb; + ASSERT_EQ(0, stat(old_path.c_str(), &sb)); + ASSERT_EQ(-1, stat(new_path.c_str(), &sb)); + + // Rename and check it moved. + ASSERT_EQ(0, rename(old_path.c_str(), new_path.c_str())); + ASSERT_EQ(-1, stat(old_path.c_str(), &sb)); + ASSERT_EQ(0, stat(new_path.c_str(), &sb)); +} + +TEST(STDIO_TEST, renameat) { + TemporaryDir td; + android::base::unique_fd dirfd{open(td.path, O_PATH)}; + std::string old_path = td.path + "/old"s; + std::string new_path = td.path + "/new"s; + + // Create the file, check it exists. + ASSERT_EQ(0, close(creat(old_path.c_str(), 0666))); + struct stat sb; + ASSERT_EQ(0, stat(old_path.c_str(), &sb)); + ASSERT_EQ(-1, stat(new_path.c_str(), &sb)); + + // Rename and check it moved. + ASSERT_EQ(0, renameat(dirfd, "old", dirfd, "new")); + ASSERT_EQ(-1, stat(old_path.c_str(), &sb)); + ASSERT_EQ(0, stat(new_path.c_str(), &sb)); +} + +TEST(STDIO_TEST, renameat2) { +#if defined(__GLIBC__) + GTEST_SKIP() << "glibc doesn't have renameat2 until 2.28"; +#else + TemporaryDir td; + android::base::unique_fd dirfd{open(td.path, O_PATH)}; + std::string old_path = td.path + "/old"s; + std::string new_path = td.path + "/new"s; + + // Create the file, check it exists. + ASSERT_EQ(0, close(creat(old_path.c_str(), 0666))); + struct stat sb; + ASSERT_EQ(0, stat(old_path.c_str(), &sb)); + ASSERT_EQ(-1, stat(new_path.c_str(), &sb)); + + // Rename and check it moved. + ASSERT_EQ(0, renameat2(dirfd, "old", dirfd, "new", 0)); + ASSERT_EQ(-1, stat(old_path.c_str(), &sb)); + ASSERT_EQ(0, stat(new_path.c_str(), &sb)); + + // After this, both "old" and "new" exist. + ASSERT_EQ(0, close(creat(old_path.c_str(), 0666))); + + // Rename and check it moved. + ASSERT_EQ(-1, renameat2(dirfd, "old", dirfd, "new", RENAME_NOREPLACE)); + ASSERT_EQ(EEXIST, errno); +#endif +} + +TEST(STDIO_TEST, renameat2_flags) { +#if defined(__GLIBC__) + GTEST_SKIP() << "glibc doesn't have renameat2 until 2.28"; +#else + ASSERT_NE(0, RENAME_EXCHANGE); + ASSERT_NE(0, RENAME_NOREPLACE); + ASSERT_NE(0, RENAME_WHITEOUT); +#endif +} diff --git a/tests/sys_mman_test.cpp b/tests/sys_mman_test.cpp index 0b981981c..e403ea5b5 100644 --- a/tests/sys_mman_test.cpp +++ b/tests/sys_mman_test.cpp @@ -230,7 +230,10 @@ TEST(sys_mman, mmap_PTRDIFF_MAX) { TEST(sys_mman, mremap_PTRDIFF_MAX) { void* map = mmap(nullptr, PAGE_SIZE, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); ASSERT_NE(MAP_FAILED, map); + ASSERT_EQ(MAP_FAILED, mremap(map, PAGE_SIZE, kHuge, MREMAP_MAYMOVE)); + + ASSERT_EQ(0, munmap(map, PAGE_SIZE)); } TEST(sys_mman, mmap_bug_27265969) { @@ -239,3 +242,61 @@ TEST(sys_mman, mmap_bug_27265969) { // Some kernels had bugs that would cause segfaults here... __builtin___clear_cache(base, base + (PAGE_SIZE * 2)); } + +TEST(sys_mman, mlock) { + void* map = mmap(nullptr, PAGE_SIZE, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + ASSERT_NE(MAP_FAILED, map); + + // Not really anything we can assert about this. + mlock(map, PAGE_SIZE); + + ASSERT_EQ(0, munmap(map, PAGE_SIZE)); +} + +TEST(sys_mman, mlock2) { +#if defined(__GLIBC__) + GTEST_SKIP() << "needs glibc 2.27"; +#else + void* map = mmap(nullptr, PAGE_SIZE, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + ASSERT_NE(MAP_FAILED, map); + + // Not really anything we can assert about this. + mlock2(map, PAGE_SIZE, MLOCK_ONFAULT); + + ASSERT_EQ(0, munmap(map, PAGE_SIZE)); +#endif +} + +TEST(sys_mman, memfd_create) { +#if defined(__GLIBC__) + GTEST_SKIP() << "needs glibc 2.27"; +#else + // Is the MFD_CLOEXEC flag obeyed? + errno = 0; + int fd = memfd_create("doesn't matter", 0); + if (fd == -1) { + ASSERT_EQ(ENOSYS, errno); + GTEST_SKIP() << "no memfd_create available"; + } + int f = fcntl(fd, F_GETFD); + ASSERT_NE(-1, f); + ASSERT_FALSE(f & FD_CLOEXEC); + close(fd); + + errno = 0; + fd = memfd_create("doesn't matter", MFD_CLOEXEC); + f = fcntl(fd, F_GETFD); + ASSERT_NE(-1, f); + ASSERT_TRUE(f & FD_CLOEXEC); + + // Can we read and write? + std::string expected("hello, world!"); + ASSERT_TRUE(android::base::WriteStringToFd(expected, fd)); + ASSERT_EQ(0, lseek(fd, 0, SEEK_SET)); + std::string actual; + ASSERT_TRUE(android::base::ReadFdToString(fd, &actual)); + ASSERT_EQ(expected, actual); + + close(fd); +#endif +} |