Compiler

RSS for tag

Discuss the various compiler and toolchain technologies used in development.

Posts under Compiler tag

64 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Illegal instruction in std::remainder (libsystem_m.dylib) when floating point exceptions enabled on M1 mac
Call to std::remainder(double(411.0), int(365)); results in a crash due to a nan in libsystem_m.dylib. MCVE program is provided + lldb backtrace and system report. $ clang++ -g -arch arm64 -std=c++20 main.cpp -o test $ ./test ori_fpcr=0, new_fpcr=1792 std::fmod(simTimeInDays, numDays) = 46 Illegal instruction: 4 main.cpp #include <cassert> #include <cfenv> #include <cmath> #include <iostream> #if !defined(__arm64__) || !defined(__APPLE__) # error "Meant to be run on arm64 apple" #endif inline int feenableexcept(unsigned int excepts) { static fenv_t fenv; if (std::fegetenv(&fenv) != 0) { return -1; } const unsigned long long old_fpcr = fenv.__fpcr; const unsigned int old_excepts = (old_fpcr >> 8u) & unsigned(FE_ALL_EXCEPT); // Check the bits passed are valid, and bit shift them const unsigned int new_excepts = excepts & unsigned(FE_ALL_EXCEPT); const unsigned long long new_fpcr = new_excepts << 8u; // Set the new bits fenv.__fpcr = fenv.__fpcr | new_fpcr; return (std::fesetenv(&fenv) != 0) ? -1 : static_cast<int>(old_excepts); } int main([[maybe_unused]] int argc, [[maybe_unused]] const char** argv) { constexpr unsigned int flags = FE_DIVBYZERO | FE_INVALID | FE_OVERFLOW; static_assert(flags == 7); constexpr uint32_t fpcr_flags_shifted = flags << 8; constexpr uint32_t fpcr_flags = (__fpcr_trap_divbyzero | __fpcr_trap_invalid | __fpcr_trap_overflow); static_assert(fpcr_flags_shifted == fpcr_flags); static_assert(fpcr_flags_shifted == 1792); uint32_t ori_fpcr = __builtin_arm_rsr("fpcr"); feenableexcept(flags); uint32_t new_fpcr = __builtin_arm_rsr("fpcr"); // std::cout << "(FE_DIVBYZERO | FE_INVALID | FE_OVERFLOW) = " << flags << '\n'; // std::cout << "((FE_DIVBYZERO | FE_INVALID | FE_OVERFLOW) << 8) = " << fpcr_flags_shifted << '\n'; // std::cout << "(__fpcr_trap_divbyzero | __fpcr_trap_invalid | __fpcr_trap_overflow) = " << fpcr_flags << '\n'; std::cout << "ori_fpcr=" << ori_fpcr << ", new_fpcr=" << new_fpcr << '\n'; const double simTimeInDays = 411.0; const int numDays = 365; // This is fine std::cout << "std::fmod(simTimeInDays, numDays) = " << std::fmod(simTimeInDays, numDays) << '\n'; // This isn't std::cout << "std::fmod(simTimeInDays, numDays) = " << std::remainder(simTimeInDays, numDays) << '\n'; return 0; } backtrace: see attachment lldb_backtrace.txt $ system_profiler SPSoftwareDataType SPHardwareDataType Software: System Software Overview: System Version: macOS 13.2 (22D49) Kernel Version: Darwin 22.3.0 Boot Volume: Macintosh HD Boot Mode: Normal Secure Virtual Memory: Enabled System Integrity Protection: Enabled Time since boot: 7 hours, 58 minutes Hardware: Hardware Overview: Model Name: MacBook Pro Model Identifier: MacBookPro18,2 Model Number: Z14V000NBFN/A Chip: Apple M1 Max Total Number of Cores: 10 (8 performance and 2 efficiency) Memory: 64 GB System Firmware Version: 8419.80.7 OS Loader Version: 8419.80.7 Activation Lock Status: Enabled $ otool -L test test: /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0 $ clang++ --version Apple clang version 14.0.0 (clang-1400.0.29.202) Target: arm64-apple-darwin22.3.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin
1
0
1.2k
Sep ’24
Cannot catch `std::invalid_argument` on clang++ under m1 -fno-rtti
Today I got stuck with a weird behaviour with my c++ project. There is the code which doesn't catch exception from standard library (the same snippet works well on intel mac and under linux machines). Code snippet: #include &lt;exception&gt; #include &lt;stdexcept&gt; #include &lt;iostream&gt; #include &lt;string&gt; int main() { try { std::stod("notanumber"); } catch (const std::invalid_argument&amp;) { std::cerr &lt;&lt; "Caught std::invalid_argument" &lt;&lt; std::endl; } } Compilation command: clang++ -fno-rtti main.cpp -o main &amp;&amp; ./main Clang versions: clang++ --version Homebrew clang version 15.0.3 Target: arm64-apple-darwin21.6.0 Thread model: posix InstalledDir: /opt/homebrew/opt/llvm/bin Also tried: Apple clang version 14.0.0 (clang-1400.0.29.102) Target: arm64-apple-darwin21.6.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin Output: libc++abi: terminating with uncaught exception of type std::invalid_argument: stod: no conversion [1] 65643 abort ./main Expected output: Caught std::invalid_argument Other info: Chip: Apple M1 OS: 12.6
4
0
1.5k
Dec ’24
The Peril of the Ampersand
A few years ago [1] Xcode added new warnings that help detect a nasty gotcha related to the lifetime of unsafe pointers. For example: Initialization of 'UnsafeMutablePointer<timeval>' results in a dangling pointer Inout expression creates a temporary pointer, but argument 'iov_base' should be a pointer that outlives the call to 'init(iov_base:iov_len:)' I’ve seen a lot of folks confused by these warnings, and by the lifetime of unsafe pointers in general, and this post is my attempt to clarify the topic. If you have questions about any of this, please put them in a new thread in the Programming Languages > Swift topic. Finally, I encourage you to watch the following WWDC presentations: WWDC 2020 Session 10648 Unsafe Swift WWDC 2020 Session 10167 Safely manage pointers in Swift These cover some of the same ground I’ve covered here, and a lot of other cool stuff as well. Share and Enjoy — Quinn “The Eskimo!” Apple Developer Relations, Developer Technical Support, Core OS/Hardware let myEmail = "eskimo" + "1" + "@apple.com" [1] Swift 5.2.2, as shipped in Xcode 11.4. See the discussion of SR-2790 in Xcode 11.4 Release Notes. Basics In Swift, the ampersand (&) indicates that a parameter is being passed inout. Consider this example: func addVarnish(_ product: inout String) { product += " varnish" } var waffle = "waffle" addVarnish(&waffle) // line A print(waffle) // printed: waffle varnish On line A, the ampersand tells you that waffle could be modified by addVarnish(_:). However, there is another use of ampersand that was designed to help with C interoperability. Consider this code: var tv = timeval() gettimeofday(&tv, nil) print(tv) // printed: timeval(tv_sec: 1590743104, tv_usec: 77027) The first parameter to gettimeofday is an UnsafeMutablePointer<timeval>. Here the ampersand denotes a conversion from a timeval to an UnsafeMutablePointer<timeval>. This conversion makes it much easier to call common C APIs from Swift. This also works for array values. For example: var hostName = [CChar](repeating: 0, count: 256) gethostname(&hostName, hostName.count) print(String(cString: hostName)) // printed: slimey.local. In this code the ampersand denotes a conversion from [CChar] to an UnsafeMutablePointer<CChar> that points to the base of the array. While this is convenient, it’s potentially misleading, especially if you come from a C background. In C-based languages, using ampersand in this way yields a pointer to the value that’s valid until the value gets deallocated. That’s not the case in Swift. Rather, the pointer generated by the ampersand syntax is only valid for the duration of that function call. To understand why that’s the case, consider this code: struct TimeInTwoParts { var sec: time_t = 0 var usec: Int32 = 0 var combined: timeval { get { timeval(tv_sec: sec, tv_usec: usec) } set { sec = newValue.tv_sec usec = newValue.tv_usec } } } var time = TimeInTwoParts() gettimeofday(&time.combined, nil) // line A print(time.combined) // printed: timeval(tv_sec: 1590743484, tv_usec: 89118) print(time.sec) // printed: 1590743484 print(time.usec) // printed: 89118 Here combined is a computed property that has no independent existence in memory. Thus, it simply makes no sense to take the address of it. So, how does ampersand deal with this? Under the covers the Swift compiler expands line A to something like this: var tmp = time.combined gettimeofday(&tmp, nil) time.combined = tmp Once you understand this it’s clear why the resulting pointer is only valid for the duration of the call: As soon as Swift cleans up tmp, the pointer becomes invalid. A Gotcha This automatic conversion can be a nasty gotcha. Consider this code: var tv = timeval() let tvPtr = UnsafeMutablePointer(&tv) // line A // ^~~~~~~~~~~~~~~~~~~~~~~~~ // Initialization of 'UnsafeMutablePointer<timeval>' results in a dangling pointer gettimeofday(tvPtr, nil) // line B This results in undefined behaviour because the pointer generated by the ampersand on line A is no longer valid when it’s used on line B. In some cases, like this one, the later Swift compiler is able to detect this problem and warn you about it. In other cases you’re not so lucky. Consider this code: guard let f = fopen("tmp.txt", "w") else { … } var buf = [CChar](repeating: 0, count: 1024) setvbuf(f, &buf, _IOFBF, buf.count) // line A let message = [UInt8]("Hello Crueld World!".utf8) fwrite(message, message.count, 1, f) // line B fclose(f) // line C This uses setvbuf to apply a custom buffer to the file handle. The file handle uses this buffer until after the close on line C. However, the pointer created by the ampersand on line A only exists for the duration of the setvbuf call. When the code calls fwrite on line B the buffer pointer is no longer valid and things end badly. Unfortunately the compiler isn’t able to detect this problem. Worse yet, the code might actually work initially, and then stop working as you change optimisation settings, update the compiler, change unrelated code, and so on. Another Gotcha There is another gotcha associated with the ampersand syntax. Consider this code: class AtomicCounter { var count: Int32 = 0 func increment() { OSAtomicAdd32(1, &count) } } This looks like it’ll implement an atomic counter but there’s no guarantee that the counter will be atomic. To understand why, apply the tmp transform from earlier: class AtomicCounter { var count: Int32 = 0 func increment() { var tmp = count OSAtomicAdd32(1, &tmp) count = tmp } } So each call to OSAtomicAdd32 could potentially be operating on a separate copy of the counter that’s then assigned back to count. This undermines the whole notion of atomicity. Again, this might work in some builds of your product and then fail in other builds. Note The above discussion is now theoretical because Swift 6 added a Synchronization module that includes comprehensive support for atomics. That module also has a Mutex type (if you need a mutex on older platforms, check out OSAllocatedUnfairLock). These constructs use various different mechanisms to ensure that the underlying value has a stable address. Summary So, to summarise: Swift’s ampersand syntax has very different semantics from the equivalent syntax in C. When you use an ampersand to convert from a value to a pointer as part of a function call, make sure that the called function doesn’t use the pointer after it’s returned. It is not safe to use the ampersand syntax for functions where the exact pointer matters. It’s Not Just Ampersands There’s one further gotcha related to arrays. The gethostname example above shows that you can use an ampersand to pass the base address of an array to a function that takes a mutable pointer. Swift supports two other implicit conversions like this: From String to UnsafePointer<CChar> — This allows you to pass a Swift string to an API that takes a C string. For example: let greeting = "Hello Cruel World!" let greetingLength = strlen(greeting) print(greetingLength) // printed: 18 From Array<Element> to UnsafePointer<Element> — This allows you to pass a Swift array to a C API that takes an array (in C, arrays are typically represented as a base pointer and a length). For example: let charsUTF16: [UniChar] = [72, 101, 108, 108, 111, 32, 67, 114, 117, 101, 108, 32, 87, 111, 114, 108, 100, 33] print(charsUTF16) let str = CFStringCreateWithCharacters(nil, charsUTF16, charsUTF16.count)! print(str) // prints: Hello Cruel World! Note that there’s no ampersand in either of these examples. This technique only works for UnsafePointer parameters (as opposed to UnsafeMutablePointer parameters), so the called function can’t modify its buffer. As the ampersand is there to indicate that the value might be modified, it’s not used in this immutable case. However, the same pointer lifetime restriction applies: The pointer passed to the function is only valid for the duration of that function call. If the function keeps a copy of that pointer and then uses it later on, Bad Things™ will happen. Consider this code: func printAfterDelay(_ str: UnsafePointer<CChar>) { print(strlen(str)) // printed: 18 DispatchQueue.main.asyncAfter(deadline: .now() + 1.0) { print(strlen(str)) // printed: 0 } } let greeting = ["Hello", "Cruel", "World!"].joined(separator: " ") printAfterDelay(greeting) dispatchMain() The second call to strlen yields undefined behaviour because the pointer passed to printAfterDelay(_:) becomes invalid once printAfterDelay(_:) returns. In this specific example the memory pointed to by str happened to contain a zero, and hence strlen returned 0 but that’s not guaranteed. The str pointer is dangling, so you might get any result from strlen, including a crash. Advice So, what can you do about this? There’s two basic strategies here: Extend the lifetime of the pointer Manual memory management Extending the Pointer’s Lifetime The first strategy makes sense when you have a limited number of pointers and their lifespan is limited. For example, you can fix the setvbuf code from above by changing it to: let message = [UInt8]("Hello Crueld World!".utf8) guard let f = fopen("tmp.txt", "w") else { … } var buf = [CChar](repeating: 0, count: 1024) buf.withUnsafeMutableBufferPointer { buf in setvbuf(f, buf.baseAddress!, _IOFBF, buf.count) fwrite(message, message.count, 1, f) fclose(f) } This version of the code uses withUnsafeMutableBufferPointer(_:). That calls the supplied closure and passes it a pointer (actually an UnsafeMutableBufferPointer) that’s valid for the duration of that closure. As long as you only use that pointer inside the closure, you’re safe! There are a variety of other routines like withUnsafeMutableBufferPointer(_:), including: The withUnsafeMutablePointer(to:_:) function The withUnsafeBufferPointer(_:), withUnsafeMutableBufferPointer(_:), withUnsafeBytes(_:), and withUnsafeMutableBytes(_:) methods on Array The withUnsafeBytes(_:) and withUnsafeMutableBytes(_:) methods on Data The withCString(_:) and withUTF8(_:) methods on String. Manual Memory Management If you have to wrangle an unbounded number of pointers — or the lifetime of your pointer isn’t simple, for example when calling an asynchronous call — you must revert to manual memory management. Consider the following code, which is a Swift-friendly wrapper around posix_spawn: func spawn(arguments: [String]) throws -> pid_t { var argv = arguments.map { arg -> UnsafeMutablePointer<CChar>? in strdup(arg) } argv.append(nil) defer { argv.forEach { free($0) } } var pid: pid_t = 0 let success = posix_spawn(&pid, argv[0], nil, nil, argv, environ) == 0 guard success else { throw NSError(domain: NSPOSIXErrorDomain, code: Int(errno), userInfo: nil) } return pid } This code can’t use the withCString(_:) method on String because it has to deal with an arbitrary number of strings. Instead, it uses strdup to copy each string to its own manually managed buffer. And, as these buffers are manually managed, is has to remember to free them. Change History 2024-12-11 Added a note about the Synchronization module. Made various editorial changes. 2021-02-24 Fixed the formatting. Added links to the WWDC 2021 sessions. Fixed the feedback advice. Minor editorial changes. 2020-06-01 Initial version.
0
0
9.7k
Dec ’24
LLVM and Terminal Support on Mac Apple Silicon
My question set is fairly broad, but I can't seem to find any answers anywhere. As a fairly low-level developer, the vast majority of my work is done with Sublime Text, terminal-based compilers, IDEs like Coq's (coq.inria.fr), and code that has to be compiled by terminal. These projects are at the very fabric of what I do, and I fear that Rosetta 2 will be inadequate until LLVM and other systems are updated to run natively on Apple Silicon. Honestly, I can't really afford complex virtualization systems like Parallels or VMWare (not that they help much) and I'd rather not give up MacOS for my Ubuntu desktop. I was raised on MacOS and I can't imagine losing features like scenes or seamless integration with the rest of my electronics. I want to keep my initial post fairly simple and broad, but if anyone has any questions on what I need supported, feel free to ask. I am sure I'm not alone here.
8
1
8k
Oct ’24