No screenshot files in XCResult files using Xcode 16.1

Hello,

I am using Xcode 16.1 (16B40) on MacOS Sequoia 15.1.0 using a Macbook pro M1 Max

I am developing an app for iOS 17 and 18 using SwiftUI I created UITests to take the screenshots for the appStore on the simulator

The tests run well and all of them are succeded

The problem appears when I try to get the screenshot files from the xcresult files after the test. There is not any screenshot inside it.

I found a data folder and a Info.plist file. In the data folder there are a lot of files with this pattern data.03zD4C6IGFFthK14NwA8mNvcwFHT16g6Tl40Tl1YmBC1bNh6d0YIcnWKyUaQPDXoa8fYo6C3Xcv8xvMtE3_NEXA== and other files with this pattern refs.03zD4C6IGFFthK14NwA8mNvcwFHT16g6Tl40Tl1YmBC1bNh6d0YIcnWKyUaQPDXoa8fYo6C3Xcv8xvMtE3_NEXA==

Ok, I tryed to use fastlane to automatize the screenshots but the problem is still present. The xcresult files have not any png file.

I had no problems doing this action (getting screenshots from a xcresult file) in previous versions of MacOS and Xcode in my current machine.

I just updated my machine to MacOS Sequoia 15.1.1 and the problem is still present

Honestly I don't know how to fix this situation.

With Xcode 15 I had not any problem with that but I am not sure if Xcode 16.0 was runing without problems because I didn't need to use this functionality in those months

Here is my code for a UITest:

import XCTest

final class ScreenshotsUITests: XCTestCase {
    let app = XCUIApplication()
    let device = "iPhone16"
    override func setUpWithError() throws {
        continueAfterFailure = true
        
    }
    
    override func tearDownWithError() throws {}
    
    @MainActor
    func testEnglishScreens() throws {
        let lang = "en"
        app.launchArguments.append("UITestMode")
        app.launchArguments += ["-AppleLanguages", "(en)"]
        app.launchArguments += ["-AppleLocale", "en_US"]
        app.launch()
        executeTestsForMenus(lang: lang, backLabel: "Back")
        executeTestForMatch(lang: lang)
    }
    
    @MainActor
    func testSpanishScreens() throws {
        let lang = "es"
        app.launchArguments.append("UITestMode")
        app.launchArguments += ["-AppleLanguages", "(es)"]
        app.launchArguments += ["-AppleLocale", "es_ES"]
        app.launch()
        executeTestsForMenus(lang: lang, backLabel: "Atrás")
        executeTestForMatch(lang: lang)
    }
    
    private func executeTestForMatch(lang: String) {
        let startButton = app.buttons["start-button"]
        startButton.tap()
        
        let key4 = app.buttons["key-4"]
        XCTAssertTrue(key4.waitForExistence(timeout: 30), "Key 4 in match screen is not found")
        key4.tap()
        let key2 = app.buttons["key-2"]
        XCTAssertTrue(key2.exists, "Key 2 in match screen is not found")
        key2.tap()
        makeScreenShot("playing", lang: lang)
        let closeButton = app.buttons["close-button"]
        XCTAssertTrue(closeButton.exists, "Close button in match screen is not found")
        closeButton.tap()
    }
    
    private func executeTestsForMenus(lang: String, backLabel: String) {
        let mainHeader = app.staticTexts["Math match"]
        XCTAssertTrue(mainHeader.exists, "Header in main screen is not found")
        makeScreenShot("mainMenu", lang: lang)
        
        let settingsButton = app.buttons["settings-button"]
        XCTAssertTrue(settingsButton.exists, "Settings button in main screen is not found")
        settingsButton.tap()
        makeScreenShot("Settings", lang: lang)
        let backButton = app.buttons[backLabel]
        XCTAssertTrue(backButton.exists, "Back button in match screen is not found")
        backButton.tap()
        
        let helpButton = app.buttons["help-button"]
        XCTAssertTrue(helpButton.exists, "Help button in main screen is not found")
        helpButton.tap()
        makeScreenShot("Help", lang: lang)
        backButton.tap()
        let scoreButton = app.buttons["score-button"]
        XCTAssertTrue(scoreButton.exists, "Scores button in main screen is not found")
        scoreButton.tap()
        makeScreenShot("Scores", lang: lang)
        backButton.tap()
        let playButton = app.buttons["play-button"]
        XCTAssertTrue(playButton.exists, "Play button in main screen is not found")
        playButton.tap()
        makeScreenShot("matchBuilder", lang: lang)
        let startButton = app.buttons["start-button"]
        XCTAssertTrue(startButton.exists, "Start button in match builder screen is not found")
    }
    
    private func makeScreenShot(_ name: String, lang: String) {
        takeScreenshot(app, named: "\(lang)-\(name)-\(device)")
    }
}

import XCTest

extension XCTestCase {
    func takeScreenshot(_ app: XCUIApplication, named name: String, fullScreen: Bool = false) {
        let screenshot: XCUIScreenshot
        if fullScreen {
            screenshot = app.windows.firstMatch.screenshot()
        } else {
            screenshot = XCUIScreen.main.screenshot()
        }
        let screenshotAttachment = XCTAttachment(
            uniformTypeIdentifier: "public.png",
            name: "screenshot-\(name).png",
            payload: screenshot.pngRepresentation,
            userInfo: nil)
        screenshotAttachment.lifetime = .keepAlways
        add(screenshotAttachment)
    }
}

and here is the content of my testplan file:

{
  "configurations" : [
    {
      "id" : "35BC7C0B-9A5A-4027-9F30-36958C4C1AAF",
      "name" : "Test Scheme Action",
      "options" : {
        "preferredScreenCaptureFormat" : "screenshot",
        "testExecutionOrdering" : "random",
        "uiTestingScreenshotsLifetime" : "keepAlways",
        "userAttachmentLifetime" : "keepAlways"
      }
    }
  ],
  "defaultOptions" : {
    "targetForVariableExpansion" : {
      "containerPath" : "container:myAppProject.xcodeproj",
      "identifier" : "B27D1B022CA00314001A259B",
      "name" : "MyAppProject"
    }
  },
  "testTargets" : [
    {
      "parallelizable" : true,
      "target" : {
        "containerPath" : "container:MyAppProject.xcodeproj",
        "identifier" : "B27D1B122CA00315001A259B",
        "name" : "MyAppProjectTests"
      }
    },
    {
      "parallelizable" : true,
      "target" : {
        "containerPath" : "container:MyAppProject.xcodeproj",
        "identifier" : "B27D1B1C2CA00315001A259B",
        "name" : "MyAppProjectUITests"
      }
    }
  ],
  "version" : 1
}

I made tests with old projects in my machine and those projects have the same problem with screenshot files in the xcresult bundles

I don't know if the problem is in my machine, my Xcode, MacOS or other ting. I don't know how to fix this problem

Please, can anyone help me?

Thanks in advance

Answered by mattk-wt in 836787022

Alright I'm back and I figured it out. Here my findings as it relates to my use case of trying to get screenshots I have saved as PNG attachments during UI tests:

  • The .xcresult bundle uses a custom storage format for attachments and other test artifacts
  • Within the result bundle's Data subdirectory, Image data (and presumably other attachment data) is stored in files whose names appear to be base64 encoded keys prefixed with "data."
  • These files are compressed using zstd. You can confirm this by running the file command on them.
  • Direct decompression of these files using zstd reveals PNG image data. Hurray!
  • As mentioned before, the filenames which are prefixed with data contain base64-encoded identifiers.
  • The database.sqlite3 database which is present in the result bundle contains a table called Attachments which has rows for each attachment.
  • Those rows have columns with the filename you provided the attachment in your test, as well as the encoded key which is appended to "data" to form the filename.
  • Those column names are name and xcResultKitPayloadRefId respectively.
  • With a little scripting you can parse the file names from the Data subdirectory, decode each file into the correct image format using zstd, look up the correct name for the file in the database using the base64 key from the original file name, and rename the newly converted PNG file accordingly.

I created a script to do all of this provided the path to the .xcresult bundle and put in a public gist on GitHub: https://gist.github.com/mhk4g/a81b16b27bfcd38a5cfc21861f171e1a

The script assumes PNG but you should be able to easily adapt it for other image formats.

Best of luck to you!

Hello again,

I could check the reports in Xcode using a shared eye (I am blind) and in the test reports the screenshots are available and I can export them but they are not present in the xcresult bundles and I cannot use automatic tools to create my screenshots for the AppStore :-(

Can anyone help me? Thanks

Hello OP, I have had some luck locating the files which I will share with you.

In my case with Xcode 16.3 and macOS 15.4 they were stored in ~/Library/Caches/com.apple.dt.Xcode/TestReport/UUID/ExtractedAttachments/ where the UUID is an identifier corresponding to this particular build/test execution.

An option to get the UUID for the most recent test run is to use ls -lrt inside that directory and the last entry would be the directory for the most recent run.

I believe it is also possible to locate the UUID by parsing the plist file found in your_derived_data_folder/your_project_folder/Logs/Test/LogStoreManifest.plist as I was able to locate the same UUID present in that plist.

Unfortunately, in my case, PNGs stored in this way did not retain the name property given to the XCUIAttachment as file names also consisted of UUIDs.

I am working on a script to automate the process. If I succeed and the result is suitable for sharing I will share it here.

I hope this helps!

Update to my previous post: unfortunately it looks like the attachments do not appear in the caches directory unless/until you view the test results within Xcode. Back to the drawing board.

Accepted Answer

Alright I'm back and I figured it out. Here my findings as it relates to my use case of trying to get screenshots I have saved as PNG attachments during UI tests:

  • The .xcresult bundle uses a custom storage format for attachments and other test artifacts
  • Within the result bundle's Data subdirectory, Image data (and presumably other attachment data) is stored in files whose names appear to be base64 encoded keys prefixed with "data."
  • These files are compressed using zstd. You can confirm this by running the file command on them.
  • Direct decompression of these files using zstd reveals PNG image data. Hurray!
  • As mentioned before, the filenames which are prefixed with data contain base64-encoded identifiers.
  • The database.sqlite3 database which is present in the result bundle contains a table called Attachments which has rows for each attachment.
  • Those rows have columns with the filename you provided the attachment in your test, as well as the encoded key which is appended to "data" to form the filename.
  • Those column names are name and xcResultKitPayloadRefId respectively.
  • With a little scripting you can parse the file names from the Data subdirectory, decode each file into the correct image format using zstd, look up the correct name for the file in the database using the base64 key from the original file name, and rename the newly converted PNG file accordingly.

I created a script to do all of this provided the path to the .xcresult bundle and put in a public gist on GitHub: https://gist.github.com/mhk4g/a81b16b27bfcd38a5cfc21861f171e1a

The script assumes PNG but you should be able to easily adapt it for other image formats.

Best of luck to you!

Nice, thanks! It works great!

Hi, I made an update to the script to make use of the first party xcrun tool xcresulttool rather than relying on parsing names from the database. I saw some instances where the sqlite database did not exist until the xcresult file was interacted with, and making use of first party tools is just a better practice in general.

The script now works on any type of attachment and takes an optional parameter for output directory.

I have replaced the old gist, the new one is here.

Cheers.

No screenshot files in XCResult files using Xcode 16.1
 
 
Q