Getting a basic URL Filter to work

I haven’t been able to get this to work at any level! I’m running into multiple issues, any light shed on any of these would be nice:

  1. I can’t implement a bloom filter that produces the same output as can be found in the SimpleURLFilter sample project, after following the textual description of it that’s available in the documentation. No clue what my implementation is doing wrong, and because of the nature of hashing, there is no way to know. Specifically:
    1. The web is full of implementations of FNV-1a and MurmurHash3, and they all produce different hashes for the same input. Can we get the proper hashes for some sample strings, so we know which is the “correct” one?
    2. Similarly, different implementations use different encodings for the strings to hash. Which should we use here?
    3. The formulas for numberOfBits and numberOfHashes give Doubles and assign them to Ints. It seems we should do this conversing by rounding them, is this correct?
    4. Can we get a sample correct value for the combined hash, so we can verify our implementations against it?
    5. Or ignoring all of the above, can we have the actual code instead of a textual description of it? 😓
  2. I managed to get Settings to register my first attempt at this extension in beta 1. Now, in beta 2, any other project (including the sample code) will redirect to Settings, show the Allow/Deny message box, I tap Allow, and then nothing happens. This must be a bug, right?
  3. Whenever I try to enable the only extension that Settings accepted (by setting its isEnabled to true), its status goes to .stopped and the error is, of course, .unknown. How do I debug this?
  4. While the extension is .stopped, ALL URL LOADS are blocked on the device. Is this to be expected? (shouldFailClosed is set to false)
  5. Is there any way to manually reload the bloom filter? My app ships blocklist updates with background push, so it would be wasteful to fetch the filter at a fixed interval. If so, can we opt out of the periodic fetch altogether?
  6. I initially believed the API to be near useless because I didn’t know of its “fuzzy matching” capabilities, which I’ve discovered by accident in a forum post. It’d be nice if those were documented somewhere!

Thanks!!

About 3):

Digging further with Console, I saw this error message:

NESMURLFilterSession[Wipr:...]: Failed to start with error: Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service created from an endpoint was invalidated from this process." UserInfo={NSDebugDescription=The connection to service created from an endpoint was invalidated from this process.}

Obviously no clue what this means 🥲 But it seems that the extensions with the NEURLFilterControlProvider cannot even be launched for some reason?

Still About 3):

While my configuration is enabled, this line keeps being spammed in the logs:

mapError unexpected error domain NEVPNConnectionErrorDomainPlugin code 7

Which appears to mean

/** @const NEVPNConnectionErrorServerDead The VPN server is no longer functioning. */
    case serverDead = 7

I don’t know if this is referring to the remote PIR server or just some locally running code.

After I disable the configuration the following is spammed like 10 times before ceasing:

mapError unexpected error domain NEVPNConnectionErrorDomainPlugin code 38

This one seems to be undocumented.

I can’t implement a bloom filter that produces the same output as can be found in the SimpleURLFilter sample project, after following the textual description of it that’s available in the documentation. No clue what my implementation is doing wrong, and because of the nature of hashing, there is no way to know. Specifically:

We are planning to publish a Bloom filter tool to help developers compose a Bloom filter with the URL data set. This will make it easier to build a Bloom filter that works with Network Extension.

I managed to get Settings to register my first attempt at this extension in beta 1. Now, in beta 2, any other project (including the sample code) will redirect to Settings, show the Allow/Deny message box, I tap Allow, and then nothing happens. This must be a bug, right?

There is a known issue that causes this behavior, we are investigating. You can try the workaround of always killing the Settings UI before enabling your URL filter, this should allow the Allow/Deny process to go through smoothly.

Whenever I try to enable the only extension that Settings accepted (by setting its isEnabled to true), its status goes to .stopped and the error is, of course, .unknown. How do I debug this?

You can enable debug logging for Network Extension (install debug logging profile), and observe the Network Extension errors. Or you can file a developer feedback with your test app and repro steps, as well as the collected sysdiagnose. Make sure to enable debug logging.

While the extension is .stopped, ALL URL LOADS are blocked on the device. Is this to be expected? (shouldFailClosed is set to false)

This sounds like a bug, if shouldFailClosed is false, when the feature fails to come up, all URLs should be allowed. Please file a feedback with repro steps and sysdiagnose.

Is there any way to manually reload the bloom filter? My app ships blocklist updates with background push, so it would be wasteful to fetch the filter at a fixed interval. If so, can we opt out of the periodic fetch altogether?

There is no manual reload for Bloom filter at this point. You can adjust the fetch interval accordingly, and return a nil Bloom filter if there is no change. When your app extension sees a change (i.e. change from your background push), then returns a new Bloom filter.

I initially believed the API to be near useless because I didn’t know of its “fuzzy matching” capabilities, which I’ve discovered by accident in a forum post. It’d be nice if those were documented somewhere!

We are updating the documentation to include the URL parsing method used by Network Extension URL Filter. Here is brief explanation, for each URL request, the system parses out the URLs into sub-URLs before performing the URL matching against the Bloom filter and if necessary, the PIR server. The parsing parses out the URL scheme, domain, port, path, query, etc and composes possible sub-URLs for matching.

The workaround of killing Settings before installing a new URL Filtering extension seems to work. However the system gets confused when you have multiple installed – if you switch one on from Settings they will all get enabled at once, which fails.

I still haven’t managed to install "debug logging for Network Extension” but I will and submit some feedbacks.

About the bloom filter tool, should we expect it in a reasonably short timeframe so we can start using it before these features ship? Cause if not I gotta go back to poking my code until I make it work right through sheer luck 😬

Getting a basic URL Filter to work
 
 
Q