Processes & Concurrency

RSS for tag

Discover how the operating system manages multiple applications and processes simultaneously, ensuring smooth multitasking performance.

Concurrency Documentation

Posts under Processes & Concurrency subtopic

Post

Replies

Boosts

Views

Activity

best practices for communication between system extension and daemon
Hello, My team has developed a DNS proxy for macOS. We have this set up with a system extension that interacts with the OS, and an always-running daemon that does all the heavy lifting. Communication between the two is DNS request and response packet traffic. With this architecture what are best practices for how the system extension communicates with a daemon? We tried making the daemon a socket server, but the system extension could not connect to it. We tried using XPC but it did not work and we could not understand the errors that were returned. So what is the best way to do this sort of thing?
3
0
681
Dec ’24
App being launched while device is locked
DESCRIPTION OF PROBLEM Logs and data from our application indicate various errors that strongly suggest that our application is being launched in a state in which the device is likely locked. We are looking for guidance on how to identify, debug, reproduce, and fix these cases. Our application does not use any of the common mechanisms for background activity, such as Background App Refresh, Navigation, Audio, etc. Errors we get in our logs such as "authorization denied (code: 23)" when trying to access a file in our app's container on disk (a simple disk cache for data our application uses) strongly suggest that the device is operating in a state, such as being locked, where our application lacks the requisite permissions it would normally have during operation. Furthermore, attempts to access authentication information stored in the keychain also fails. We use kSecAttrAccessibleWhenUnlocked when accessing items we store in the keychain. We have investigated "Prewarming", as well as our notification extension that helps process incoming push notifications, but cannot find any way to recreate this behavior. Are there any steps Apple engineers can recommend to triage and debug this? Some additional questions that would help us: What are all of the symptoms that we can look for if prewarming escapes the intended execution context? What are all of the circumstances in which we would be unauthorized to access the app’s documents/file directories even if it works correctly in normal operation? STEPS TO REPRODUCE Unfortunately, we are unable to forcibly reproduce this behavior in our application, so we're looking for guidance on how we might simulate this behavior in Xcode / Instruments. Are there tools that Apple provides that would allow us to simulate certain behaviors like prewarming to verify our application's functionality? Are there other reasons our application might be launched while the device is locked? Are there other reasons we would receive security errors when accessing the keychain or disk that are unrelated to the device being locked?
1
1
493
Jan ’25
Background refresh or processing app
I am writing an app which mainly is used to update data used by other apps on the device. After the user initializes some values in the app, they almost never have to return to it (occasionally to add a "friend"). The app needs to run a background task at least daily, however, without the user's intervention (or even awareness, once they've given permission). My understanding of background refresh tasks is that if the user doesn't activate the app in the foreground periodically, the scheduled background tasks may never run. If this is true, do I want to use a background processing task instead, or is there a better solution (or have I misunderstood entirely)?
1
0
363
Jan ’25
How to debug or ensure single resume calls for continuations at compile time or with tools like Instruments?
When using the continuation API, we're required to call resume exactly once. While withCheckedContinuation helps catch runtime issues during debugging, I'm looking for ways to catch such errors at compile time or through tools like Instruments. Is there any tool or technique that can help enforce or detect this requirement more strictly than runtime checks? Or would creating custom abstractions around Continuation be the only option to ensure safety? Any suggestions or best practices are appreciated.
2
0
394
Jan ’25
Some fundamental doubts about DisptachQueue and GCD
I understand that GCD and it's underlying implementations have evolved over time. And many things have not been shared explicitly in Apple documentation. The most concepts of DispatchQueue (serial and concurrent queues), DispatchQoS, target queue and system provided queues: main and globals etc. I have some doubts & questions to clarify: [Main Dispatch Queue] [Link] Because the main queue doesn't behave entirely like a regular serial queue, it may have unwanted side-effects when used in processes that are not UI apps (daemons). For such processes, the main queue should be avoided. What does it mean? Can you elaborate? [Global Concurrent Dispatch Queues] Are they global to a process or across processes on a device. I believe it is the first case but just wanted to be sure. [Global Concurrent Dispatch Queues] Does system create 4 (for each QoS) * 2 (over-commiting and non-overcommiting queues) = 8 queues in all. When does which type of queue comes into play? [Custom Queue][Target Queue concept] [swift-corelibs-libdispatch/man/dispatch_queue_create.3] QUOTE The default target queue of all dispatch objects created by the application is the default priority global concurrent queue. UNQUOTE Is this stil true? We could not find a mention of this in any latest official apple documentation (though some old forum threads (one more) and github code documentation indicate the same). The official documentation only says: [dispatch_set_target_queue] QUOTE If you want the system to provide a queue that is appropriate for the current object UNQUOTE [dispatch_queue_create_with_target] QUOTE Specify DISPATCH_TARGET_QUEUE_DEFAULT to set the target queue to the default type for the current dispatch queue.UNQUOTE [Dispatch>DispatchQueue>init] QUOTE Specify DISPATCH_TARGET_QUEUE_DEFAULT if you want the system to provide a queue that is appropriate for the current object. UNQUOTE What is the difference between passing target queue as 'nil' vs 'DISPATCH_TARGET_QUEUE_DEFAULT' to DispatchQueue init? [Custom Queue][Target Queue concept] [dispatch_set_target_queue] QUOTE The system doesn't allocate threads to the dispatch queue if it has a target queue, unless that target queue is a global concurrent queue. UNQUOTE The system does allocate threads to the custom dispatch queues that have global concurrent queue as the default target. What does that mean? Why does targetting to global concurrent queues mean in that case? [System / GCD Thread Pool] that excutes work items from DispatchQueue: Is this thread pool per queue? or across queues per process? or across processes per device?
14
0
1.1k
Jan ’25
What DispatchQueues should i use for my app's communication subsystem?
We would be creating N NWListener objects and M NWConnection objects in our process' communication subsystem to create server sockets, accepted client sockets on server and client sockets on clients. Both NWConnection and NWListener rely on DispatchQueue to deliver state changes, incoming connections, send/recv completions etc. What DispatchQueues should I use and why? Global Concurrent Dispatch Queue (and which QoS?) for all NWConnection and NWListener One custom concurrent queue (which QoS?) for all NWConnection and NWListener? (Does that anyways get targetted to one of the global queues?) One custom concurrent queue per NWConnection and NWListener though all targetted to Global Concurrent Dispatch Queue (and which QoS?)? One custom concurrent queue per NWConnection and NWListener though all targetted to single target custom concurrent queue? For every option above, how am I impacted in terms of parallelism, concurrency, throughput & latency and how is overall system impacted (with other processes also running)? Seperate questions (sorry for the digression): Are global concurrent queues specific to a process or shared across all processes on a device? Can I safely use setSpecific on global dispatch queues in our app?
13
0
868
Jan ’25
Operation not permitted on xpc_listener_create
Hi, I'm trying to create a launch daemon that uses XPC to receive requests from an unprivileged app. Ultimately both components will be written in Go. For now I'm trying to write a PoC in Objective-C to make sure I get everything right, so I'm compiling / signing from the CLI, and writing plist files by hand -- I'm not using XCode. My current daemon code is pretty much the same as the boilerplate code that XCode generates when creating a new 'XPC Service': #import <stdio.h> #include <xpc/xpc.h> int main(int argc, char *argv[]) { xpc_rich_error_t error; dispatch_queue_t queue = dispatch_queue_create("com.foobar.daemon", DISPATCH_QUEUE_SERIAL); xpc_listener_t listener = xpc_listener_create( "com.foobar.daemon", queue, XPC_LISTENER_CREATE_NONE, ^(xpc_session_t _Nonnull peer) { xpc_session_set_incoming_message_handler(peer, ^(xpc_object_t _Nonnull message) { int64_t firstNumber = xpc_dictionary_get_int64(message, "firstNumber"); int64_t secondNumber = xpc_dictionary_get_int64(message, "secondNumber"); // Create a reply and send it back to the client. xpc_object_t reply = xpc_dictionary_create_reply(message); xpc_dictionary_set_int64(reply, "result", firstNumber + secondNumber); xpc_rich_error_t replyError = xpc_session_send_message(peer, reply); if (replyError) { printf("Reply failed, error: %s", xpc_rich_error_copy_description(replyError)); } }); }, &error); if (error != NULL) { printf("ERROR: %s\n", xpc_rich_error_copy_description(error)); exit(1); } printf("Created listener: %s", xpc_listener_copy_description(listener)); // Resuming the serviceListener starts this service. This method does not return. dispatch_main(); return 0; } I'm compiling, signing and installing my daemon with the following commands: build_foobar() { clang -Wall -x objective-c -o com.foobar.daemon poc/main.m codesign --force --verify --verbose --options=runtime \ --identifier="com.foobar.daemon" \ --sign="Mac Developer: Albin Kerouanton (XYZ)" \ --entitlements=poc/entitlements.plist \ com.foobar.daemon } install_foobar() { sudo cp com.foobar.daemon /Library/PrivilegedHelperTools/com.foobar.daemon sudo cp poc/com.foobar.daemon.plist /Library/LaunchDaemons/com.foobar.daemon.plist sudo launchctl bootout system/com.foobar.daemon || true sudo launchctl bootstrap system /Library/LaunchDaemons/com.foobar.daemon.plist } Here's the content of my entitlements.plist file: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>com.apple.application-identifier</key> <string>ABCD.com.foobar.daemon</string> </dict> </plist> And finally, here's my launchd plist file: <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>com.foobar.daemon</string> <key>Program</key> <string>/Library/PrivilegedHelperTools/com.foobar.daemon</string> <key>ProgramArguments</key> <array> <string>/Library/PrivilegedHelperTools/com.foobar.daemon</string> </array> <key>RunAtLoad</key> <false/> <key>StandardOutPath</key> <string>/tmp/com.foobar.daemon.out.log</string> <key>StandardErrorPath</key> <string>/tmp/com.foobar.daemon.err.log</string> <key>Debug</key> <true/> </dict> </plist> Whenever I start my service using sudo launchctl start com.foobar.daemon, it exits with the following error message: ERROR: Unable to activate listener: failed at listener activation with error 1 - Operation not permitted System logs don't show anything interesting -- they're just repeating the same error message. I tried to add / remove some properties from both the entitlement and the launchd plist file but to no avail. Any idea what's going wrong?
1
0
483
Jan ’25
Schedule BGAppRefreshTask more often for debugging purposes
I am considering to use the BGAppRefreshTask mechanism, and while I think I have read and understood all documentation and hints in this forum about it (especially the limitations), the one thing I do not understand is: how can I debug it? I cannot find a way to trigger the BGAppRefreshTask execution reliably and immediately. I would have expected the Xcode Debug->Simulate Background Fetch menu to do this for me, but it only sends the app into the background. I am working with the unmodified (except for a few added print()) ColorFeed sample code project from Apple, which schedules a task 15min into the future when it goes to the background. Using a real device, I have not managed to trigger execution of the BGAppRefreshTask more often than once a day so far. Surely, there must be a way to trigger it much more often solely for debugging and development purposes (I am totally happy with all restrictions for the final app). So what detail am I missing here?
1
0
503
Jan ’25
XPC between main app and system extension
Hello, I'm developing a Mac application that uses a network extension. I'm trying to implement XPC to pass data between my main app and system extension and I'm using the SimpleFirewall demo app as a guide to do this. One thing I can't understand is how the ViewController in the SimpleFirewall main app has access to the class IPCConnection in the SimpleFirewallExtension without it being public and without SimpleFirewallExtension being imported in ViewController.
1
0
487
Jan ’25
Check whether XPC remote proxy responds to selector, without causing exception and connection invalidation?
I have several processes maintaining NSXPConnection to an XPC service. The connections are bi-directional. Each side service and clients) of the connection exports an object, and an XPCInterface. The @protocols are different - to the service, and from the service to clients. So long as all the "clients" fully implement their "call-back" @protocol, there's no problem. All works fine. However - If a client does NOT implement a method in the "call back protocol", or completely neglects to export an object, or interface - and the service attempts to call back using the nonexistent method -- the XPC connection invalidates immediately. So far - expected behaviour. However, if I want the service to behave to the client a little like a "delegate" style -- and check first whether the client "respondsToSelector" or even - supports an interface BEFORE calling it, then this doesn't work. When my XPC service tries the following on a client connection: if (xpcConnection.remoteObjectInterface == nil) os_log_error(myXPCLog, "client has no remote interface); the condition is never met - i.e. the "remoteObjectInterface is never nil even when the client does NOT configure its NSXPCConnection with any incoming NSXPCInterface, and does not set an "exportedObject" Furthermore, the next check: if ([proxy respondsToSelector:@selector(downloadFiltersForCustomer:withReply:)]) { } will not only fail - but will drop the connection. The client side gets the invalidation with the following error: <NSXPCConnection: 0x600000b20000> connection to service with pid 2477 named com.proofpoint.ecd: received an undecodable message for proxy 1 (no exported object to receive message). Dropping message. I guess the "undecidable message" is the respondsToSelector - because the code doesn't get to attempt anything else afterwards, the connection drops. Is there a way to do this check "quietly", or suffering only "interruption", but without losing the connection,
3
0
659
Jan ’25
Using protocols with XPC C API instead of dictionaries for sending and receiving messages
I have followed this post for creating a Launch Agent that provides an XPC service on macOS using Swift- post link - https://rderik.com/blog/creating-a-launch-agent-that-provides-an-xpc-service-on-macos/ In the swift code the interface of the XPC service is defined by protocols which makes the code nice and neat. I want to implement the XPC service using C APIs for XPC, and C APIs send and receive messages using dictionaries, which need manual handling with conditional statements. I want to know if its possible to go with the protocol based approach with C APIs.
2
0
415
Jan ’25
Keeping a socket connection active in app extension in a Safari Web Extension
I’m currently porting a Chrome Extension to Safari and integrating it with native messaging in a Safari Web Extension. As part of this, I’m building a proxy to forward messages between the web extension and a socket in another application, both ways. Additionally, the socket occasionally broadcasts messages that also need to be sent to the web extension. The issue I’m facing is that the app extension terminates whenever I call context.completeRequest(returningItems: nil), which prevents me from listening for incoming messages from the socket (I'm using the Network Framework). To work around this, I’ve tried not calling context.completeRequest(returningItems: nil), which keeps the app extension running. However, I’m unsure if this is the right approach—currently, I’m simply ignoring the response and relying entirely on SFSafariApplication.dispatchMessage. According to the documentation, the app extension lifecycle ends when the system terminates it, but I need to keep the socket listener active. Has anyone encountered a similar issue, or does anyone have suggestions for maintaining the socket connection while adhering to the app extension lifecycle? Any insights would be greatly appreciated!
2
0
452
Jan ’25
Handling XPC Communication to Multiple Clients: Is Storing Connections a Reliable Approach?
This is the functionality I am trying to achieve with libxpc: There's one xpc server and two xpc clients. When the xpc server receives a particular dictionary item from clientB, the server needs to send a response to both clientA and clientB. This is the approach I am currently using: First, clientA creates a dictionary item that indicates that this item is from clientA. Now, clientA sends this dictionary to server. When server receives this item, it stores the connection instance with clientA in a global variable. Next, when clientB sends a particular dictionary item, server uses this global variable where it perviously stored clientA's connection instance to send a response back to clientA, alongside clientB. Only one edge case I can see is that when clientA closes this connection instance, server will be trying to send a response to an invalidated connection. Question: Is this approach recommended? Any edge cases I should be aware of? Is there any better way to achieve this functionality?
2
0
412
Jan ’25
HELP: Privileged Helper With SMAppService
Hi! I've been developing iOS and macOS apps for many years, but now I am looking to dive into smth i have never touched before, namely privileged helpers, and i am struggling hard trying to find my footing. Here’s my use case: I have a CLI tool that requires elevated privileges. I want to create a menu bar app that can interact with this tool, but I’m struggling to find solid documentation or examples of how to accomplish this using SMAppService. I might just be missing something obvious. If anyone could point me toward relevant documentation, examples, articles, tutorials, or even a WWDC session that covers running privileged helpers with SMAppService, I would greatly appreciate it. Thanks in advance!
1
0
402
Feb ’25
Launch constraints using LightweightCodeRequirements framework
MacOS Version: 14.7.2 macOS SDKs: macOS 14.5 -sdk macosx14.5 I am working on a sample program for validation Against: Team Identifier Developer ID I started with validating Team Identifier, but my validation is not working and it is allowing to launch programs which are not matching the team identifier in the signature. Below is my code: func verifyExecutableWithLCR(executablePath: String, arguments: [String]) -&gt; Bool { let task = Process() task.launchPath = executablePath task.arguments = arguments if #available(macOS 14.4, *) { print("launchRequirementData is available on this system.") do { let req = try OnDiskCodeRequirement.allOf { TeamIdentifier("ABCDEFGHI") //SigningIdentifier("com.***.client.***-Client.****") } let encoder = PropertyListEncoder() encoder.outputFormat = .xml let requirementData = try encoder.encode(req) task.launchRequirementData = requirementData print("launchRequirementData is set.") try task.run() print("[SUCCESS] Executable passed the code signature verification.") return true } catch { print("[ERROR] Code signature verification failed: \(error.localizedDescription)") return false } } else { print("[WARNING] launchRequirement is not available on this macOS version.") return false } } Could you please help me in identifying whay am I doing wrong here?
7
0
501
Feb ’25
Stuck threads in Endpoint Security extension
I have a weird issue with our Endpoint Security extension. A couple of the checks we do require calling into Apple frameworks (Security, Disk Arbitration) and these checks can happen early in the boot process. On macOS 13 (and possibly earlier), sometimes these calls get stuck and never return. When this happens, the kernel will kill the extension and this generates a crash log. This adds significant time to the boot, enough that people notice. In every case, the thread where the call into the Apple framework occurred shows that the thread is stuck in mach_msg2_trap(), which I now understand means it's likely waiting on an event or message. Now here's where things get weird. I discovered that if I shunt the check off onto a Thread subclass and put it in a DispatchGroup (perhaps the wrong primitive), then wait() on that group with my own timeout, the thread will get unstuck within a couple hundred milliseconds of the timeout. The timeout can be a couple of seconds or longer. In every case, the thread unblocks, returns from mach_msg2_trap() and the original call finishes as expected. Is there a rational explanation for this behavior? Am I crazy to even consider shipping this workaround?
3
0
467
Feb ’25