Accessing security scoped URLs without calling url.startAccessingSecurityScopedResource

I have discovered a gap in my understanding of user selected URLs in iOS, and I would be grateful if someone can put me right please.

My understanding is that a URL selected by a user can be accessed by calling url.startAccessingSecurityScopedResource() call. Subsequently a call to stopAccessingSecurityScopedResource() is made to avoid sandbox memory leaks.

Furthermore, the URL can be saved as a bookmark and reconstituted when the app is run again to avoid re-asking permission from the user.

So far so good.

However, I have discovered that a URL retrieved from a bookmark can be accessed without the call to url.startAccessingSecurityScopedResource(). This seems contrary to what the documentation says here

So my question is (assuming this is not a bug) why not save and retrieve the URL immediately in order to avoid having to make any additional calls to url.startAccessingSecurityScopedResource?

Bill Aylward

You can copy and paste the code below into a new iOS project to illustrate this. Having chosen a folder, the 'Summarise folder without permission' button fails as expected, but once the 'Retrieve URL from bookmark' has been pressed, it works fine.

import SwiftUI
import UniformTypeIdentifiers

struct ContentView: View {
	
	@AppStorage("bookmarkData") private var bookmarkData: Data?
	@State private var showFolderPicker = false
	@State private var folderUrl: URL?
	@State private var folderReport: String?
	
	var body: some View {
		VStack(spacing: 20) {
			Text("Selected folder: \(folderUrl?.lastPathComponent ?? "None")")
			
			Text("Contents: \(folderReport ?? "Unknown")")
			
			Button("Select folder") {
				showFolderPicker.toggle()
			}
			
			Button("Deselect folder") {
				folderUrl = nil
				folderReport = nil
				bookmarkData = nil
			}
			.disabled(folderUrl == nil)
			
			Button("Retrieve URL from bookmark") {
				retrieveFolderURL()
			}
			.disabled(bookmarkData == nil)
			
			Button("Summarise folder with permission") {
				summariseFolderWithPermission(true)
			}
			.disabled(folderUrl == nil)
			
			Button("Summarise folder without permission") {
				summariseFolderWithPermission(false)
			}
			.disabled(folderUrl == nil)
		}
		.padding()
		.fileImporter(
			isPresented: $showFolderPicker,
			allowedContentTypes: [UTType.init("public.folder")!],
			allowsMultipleSelection: false
		) { result in
			switch result {
			case .success(let urls):
				if let selectedUrl = urls.first {
					print("Processing folder: \(selectedUrl)")
					processFolderURL(selectedUrl)
				}
			case .failure(let error):
				print("\(error.localizedDescription)")
			}
		}
		.onAppear() {
			guard folderUrl == nil else { return }
			retrieveFolderURL()
		}
	}
	
	func processFolderURL(_ selectedUrl: URL?) {
		
		guard selectedUrl != nil else { return }
		
		// Create and save a security scoped bookmark in AppStorage
		do {
			guard selectedUrl!.startAccessingSecurityScopedResource() else { print("Unable to access \(selectedUrl!)"); return }
			
			// Save bookmark
			bookmarkData = try selectedUrl!.bookmarkData(options: .minimalBookmark, includingResourceValuesForKeys: nil, relativeTo: nil)

			selectedUrl!.stopAccessingSecurityScopedResource()
			
		} catch {
			print("Unable to save security scoped bookmark")
		}
		
		folderUrl = selectedUrl!
	}
	
	func retrieveFolderURL() {
		guard let bookmarkData = bookmarkData else {
			print("No bookmark data available")
			return
		}
		
		do {
			var isStale = false
			let url = try URL(
				resolvingBookmarkData: bookmarkData,
				options: .withoutUI,
				relativeTo: nil,
				bookmarkDataIsStale: &isStale
			)
			
			folderUrl = url
			
		} catch {
			print("Error accessing URL: \(error.localizedDescription)")
		}
	}
	
	func summariseFolderWithPermission(_ permission: Bool) {
		
		folderReport = nil
		print(String(describing: folderUrl))
		
		guard folderUrl != nil else { return }
		if permission { print("Result of access requrest is \(folderUrl!.startAccessingSecurityScopedResource())") }
		
		do {
			let contents = try FileManager.default.contentsOfDirectory(atPath: folderUrl!.path)
			folderReport = "\(contents.count) files, the first is: \(contents.first!)"
			
		} catch {
			print(error.localizedDescription)
		}
		
		if permission { folderUrl!.stopAccessingSecurityScopedResource() }
	}
}
Answered by DTS Engineer in 849593022

I have discovered a gap in my understanding of user-selected URLs in iOS, and I would be grateful if someone can put me right, please.

I will do my best, but I'll freely admit that the behaviors here are more than a little... opaque.

However, I have discovered that a URL retrieved from a bookmark can be accessed without the call to url.startAccessingSecurityScopedResource().

Yes, that is correct. URLs can have security scope "attached" to them, and on iOS, it's very difficult (impossible?) to get a URL out of ANY API that doesn't have a security scope attached (this is also called "implicit" scope vs the "explicit" scope that start/stop manage). In theory, we should someday be able to turn off implicit scope, at which point start/stop WOULD be required...

This seems contrary to what the documentation says here.

Well... that's not quite true, but it takes a VERY close reading of the documentation to catch it. What the documentation actually says is:

"When you obtain a security-scoped URL, such as by resolving a security-scoped bookmark, you can’t immediately use the resource it points to."

The catch here is that you did NOT resolve a security-scoped bookmark. You create a security-scoped bookmark by using NSURLBookmarkCreationWithSecurityScope,, but you couldn't do that because that symbol doesn't exist on iOS. What you created was a "regular" bookmark which also happened to have enough data embedded to restore implicit security scope. As the saying goes, "technically correct" is the best kind... Though this is definitely not the most successful part of our API or documentation.

There's actually a hint of this dynamic in the documentation, since "NSURLBookmarkCreationWithoutImplicitSecurityScope" IS defined on iOS. I can't think of any reason why you'd use that option, but if you did, then you wouldn't be able to use the URL after it was resolved*.

*Probably? The behavior here is very messy as this is actually managed at the process level, not the individual URL level.

In any case, the practical consequence of all this is:

  • It's good practice to call start/stop, since we might somebody drop or restrict implicit scope.

  • It's very difficult to "get" a URL from the system that you can't actually access, which means start/stop aren't as essential as they would be on iOS.

So my question is (assuming this is not a bug) why not save and retrieve the URL immediately in order to avoid having to make any additional calls to url.startAccessingSecurityScopedResource?

I haven't benchmarked it, but I suspect it's considerably slower, as you're basically doing everything startAccessingSecurityScopedResource, plus a bunch of extra I/O calls to first generate and then resolve the bookmark.

Furthermore, the URL can be saved as a bookmark and reconstituted when the app is run again to avoid re-asking permission from the user.

Note that there is an iOS bug (r.102995804) that means bookmarks to any external volume break as soon as the volume is remounted. Unfortunately, that means that the kinds of apps that would most benefit from persistent access (because they want to do bulk I/O to external targets) can't persist access. If this is an issue for your app, please file a bug on this that describes exactly why this is important to your app and post the bug number back here. I would love to get this fixed, but there have been surprisingly few reports about it.

Thanks for your reply, but I remain confused about this. When I run my code on a real device, the 'Summarise folder without permission' button does NOT work, so there doesn't seem to be any implicit "start". Strangely running the identical code on a simulator DOES work, which I also find confusing.

Ironically, I believe this is actually caused by a shortcut the simulator uses to behave more like iOS. Instead of trying to replicate the implicit system iOS relies so heavily on, I believe there is a broader sandbox exception granted to your app’s simulator process. This ensures that the URLs the simulator returns work while avoiding all the work of trying to replicate what iOS is doing. It also means your simulator process can access places it "shouldn't" be able to, but that's not really an issue given what the simulator is "for".

Regarding performance, my interest in this started with sandbox_extension_consume error=[12: Cannot allocate memory] errors occurring when accessing hundreds of URLs, each bracketed by the accessing calls.

Can you tell me more about this? Is this on iOS or the simulator? What are you doing that's triggering this?

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

This seems contrary to what the documentation says here

That documentation specifically refers to a security-scoped URL. That is a URL that you've retrieved from a security-scoped bookmark. If you've retrieved the URL from some other means, then these methods may not necessary (but see below).

why not save and retrieve the URL immediately in order to avoid having to make any additional calls to url.startAccessingSecurityScopedResource?

Saving and retrieving a URL is a time-consuming hassle, especially compared to calling these two methods. You only want to do that if you really need to.

You can call start/stopAccessingSecurityScopedResource on any URL. You just have to pay attention to the result from startAccessingSecurityScopedResource. You only need to call stop if start returned true.

When you get a URL via the standard UI pickers, then it already has an implicit "start". That's why it works. There's no problem with calling "start" again, as long as you don't call "stop" if "start" returns false. This is handy for cases where you do the URL handling elsewhere. You can just always call start/stop, and if you do it correctly, it will always work.

In some cases, especially on macOS, saving and loading bookmarks can be more tricky, so that's another reason to avoid it if you don't need to do that.

Thanks for your reply, but I remain confused about this. When I run my code on a real device, the 'Summarise folder without permission' button does NOT work, so there doesn't seem to be any implicit "start". Strangely running the identical code on a simulator DOES work, which I also find confusing.

Regarding performance, my interest in this started with sandbox_extension_consume error=[12: Cannot allocate memory] errors occurring when accessing hundreds of URLs, each bracketed by the accessing calls.

A single save and retrieve when the user chooses the folder seems not too much hassle compared to this.

However, my main concern is that the code doesn't seem to be behaving as expected from the documentation, and I don't understand why!

the 'Summarise folder without permission' button does NOT work, so there doesn't seem to be any implicit "start"

I'm sorry, but I don't know what you mean by "does NOT work". Is it throwing an exception? Or failing in some other way? What is the specific context of this "failure"?

You aren't checking the result of startAccessingSecurityScopedResource(). So then you are calling stopAccessingSecurityScopedResource() based only on the permission value, which is wrong. You should only call stopAccessingSecurityScopedResource() if startAccessingSecurityScopedResource() returns true.

Strangely running the identical code on a simulator DOES work, which I also find confusing.

That part's easy enough. Whenever there is a discrepancy between the behaviour of the simulator vs. a real device, the simulator is always wrong.

errors occurring when accessing hundreds of URLs, each bracketed by the accessing calls

I don't know how you're ever going to get hundreds of URLs on iOS. Normally, the user selects a single URL. If you are loading and saving bookmarks, you're probably only doing that for a few URLs at a time. It makes sense to put the accessing calls at a higher level of code. That way, when your lower-level code accesses the contents, perhaps with hundreds of URLs, you don't have to worry about any of this. Those are then just regular 'ole URLs.

my main concern is that the code doesn't seem to be behaving as expected from the documentation, and I don't understand why!

As I said, the documentation is very explicit about when these accessing calls are required. You only need them when loading from a security-scoped bookmark. In your case, because you need them when loading from a security-scoped bookmark, they work, even though you aren't using them correctly. In this case, it just so happens that startAccessingSecurityScopedResource() returns true and then your call to stopAccessingSecurityScopedResource() also works. But then when you use them incorrectly in the case where you don't need them, they fail, because you haven't checked the result of startAccessingSecurityScopedResource() and you call stopAccessingSecurityScopedResource() when you shouldn't.

My recommendation is to simply use the API correctly and then everything works. That way, your lower-level code never needs to know about any of this. Your code is also portable across other Apple platforms like macOS where these bookmarks behave differently. And your code is more robust because it will then work regardless of where the URL came from. You can use the same business logic for a URL read from a file or defaults, or selected by the user.

Accepted Answer

I have discovered a gap in my understanding of user-selected URLs in iOS, and I would be grateful if someone can put me right, please.

I will do my best, but I'll freely admit that the behaviors here are more than a little... opaque.

However, I have discovered that a URL retrieved from a bookmark can be accessed without the call to url.startAccessingSecurityScopedResource().

Yes, that is correct. URLs can have security scope "attached" to them, and on iOS, it's very difficult (impossible?) to get a URL out of ANY API that doesn't have a security scope attached (this is also called "implicit" scope vs the "explicit" scope that start/stop manage). In theory, we should someday be able to turn off implicit scope, at which point start/stop WOULD be required...

This seems contrary to what the documentation says here.

Well... that's not quite true, but it takes a VERY close reading of the documentation to catch it. What the documentation actually says is:

"When you obtain a security-scoped URL, such as by resolving a security-scoped bookmark, you can’t immediately use the resource it points to."

The catch here is that you did NOT resolve a security-scoped bookmark. You create a security-scoped bookmark by using NSURLBookmarkCreationWithSecurityScope,, but you couldn't do that because that symbol doesn't exist on iOS. What you created was a "regular" bookmark which also happened to have enough data embedded to restore implicit security scope. As the saying goes, "technically correct" is the best kind... Though this is definitely not the most successful part of our API or documentation.

There's actually a hint of this dynamic in the documentation, since "NSURLBookmarkCreationWithoutImplicitSecurityScope" IS defined on iOS. I can't think of any reason why you'd use that option, but if you did, then you wouldn't be able to use the URL after it was resolved*.

*Probably? The behavior here is very messy as this is actually managed at the process level, not the individual URL level.

In any case, the practical consequence of all this is:

  • It's good practice to call start/stop, since we might somebody drop or restrict implicit scope.

  • It's very difficult to "get" a URL from the system that you can't actually access, which means start/stop aren't as essential as they would be on iOS.

So my question is (assuming this is not a bug) why not save and retrieve the URL immediately in order to avoid having to make any additional calls to url.startAccessingSecurityScopedResource?

I haven't benchmarked it, but I suspect it's considerably slower, as you're basically doing everything startAccessingSecurityScopedResource, plus a bunch of extra I/O calls to first generate and then resolve the bookmark.

Furthermore, the URL can be saved as a bookmark and reconstituted when the app is run again to avoid re-asking permission from the user.

Note that there is an iOS bug (r.102995804) that means bookmarks to any external volume break as soon as the volume is remounted. Unfortunately, that means that the kinds of apps that would most benefit from persistent access (because they want to do bulk I/O to external targets) can't persist access. If this is an issue for your app, please file a bug on this that describes exactly why this is important to your app and post the bug number back here. I would love to get this fixed, but there have been surprisingly few reports about it.

Thanks for your reply, but I remain confused about this. When I run my code on a real device, the 'Summarise folder without permission' button does NOT work, so there doesn't seem to be any implicit "start". Strangely running the identical code on a simulator DOES work, which I also find confusing.

Ironically, I believe this is actually caused by a shortcut the simulator uses to behave more like iOS. Instead of trying to replicate the implicit system iOS relies so heavily on, I believe there is a broader sandbox exception granted to your app’s simulator process. This ensures that the URLs the simulator returns work while avoiding all the work of trying to replicate what iOS is doing. It also means your simulator process can access places it "shouldn't" be able to, but that's not really an issue given what the simulator is "for".

Regarding performance, my interest in this started with sandbox_extension_consume error=[12: Cannot allocate memory] errors occurring when accessing hundreds of URLs, each bracketed by the accessing calls.

Can you tell me more about this? Is this on iOS or the simulator? What are you doing that's triggering this?

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks very much indeed for your detailed explanation, and I am much clearer now about what is going on.

In answer to you question, I have a marine navigation app which has a function "openChart(url)" to open a file stored on the device in a user selected folder, and the function brackets the file opening code with a start/stop.

When the user selects a folder, the code loops through the contents and does two things. 1. Uses FileManager to obtain the file size and creation date (bracketed by a start/stop) 2. Calls openChart(url) to extract a title and create a thumbnail

So that's 2 start/stop calls per loop, which was actually fine until very recently when a customer selected a folder with 764 files in it. The error appeared after exactly 340 iterations on his device as well as on my iPad pro, and could only be cured by re-starting the app.

Saving and loading the bookmark, along with removing the start/stop calls solved the problem, but following your explanation I have implemented the suggestion made by Etresoft and moved the start/stop code to bracket the loop. This also fixes the problem.

I haven't so far encountered a need to use an external volume so have not yet encountered bug r.102995804. However, it is a possible use case for my app so I will look into that and report back as you suggest.

So that's 2 start/stop calls per loop, which was actually fine until very recently when a customer selected a folder with 764 files in it.

Ahh. Did you try just calling start/stop on the original directory and not on each individual file? Security scope applies to directories as well, so I don't think you actually need to do this for each file.

Also, on the number here:

The error appeared after exactly 340 iterations on his device as well as on my iPad Pro, and could only be cured by re-starting the app.

That's actually much lower than I'd have expected. It's possible the limits are significantly lower on iOS, but there may also have been something else going on.

Covering a few other details:

  1. Uses FileManager to obtain the file size and creation date (bracketed by a start/stop)

FYI, I don't think this ever actually requires calling start (on the specific file), even on macOS, as doing so would severely compromise many of our APIs. Notably, directory access APIs like contentsOfDirectory(at:URL...) and enumerator(at:URL...) rely on getattrlistbulk() to retrieve the file name and requested metadata of multiple files in a single path*. The sandbox can prevent you from retrieving the contents of a directory, but if it allows you to access the contents it will also allow you to retrieve all of the standard file metadata. More to the point, if you used the APIs above (which you should have been...), then the metadata you were asking for should already have been cached as part of the URL you retrieved.

*This is why these APIs include a "includingPropertiesForKeys" argument- it tells the API exactly what metadata getattrlistbulk() should request and the data returned is then "attached" the the URL that's returned.

Saving and loading the bookmark, along with removing the start/stop calls solved the problem,

It would take a very deep dive into the code to confirm this, but I suspect this worked because the system applied the sandox extension you already had from the original directory instead of creating a new extension.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Many thanks Kevin, Calling start/stop on the enclosing directory also fixes the issue.

Accessing security scoped URLs without calling url.startAccessingSecurityScopedResource
 
 
Q