Ducking MusicKit output when playing another sound

I am developing an app that uses MusicKit to play music and then I need to have spoken words played to the user, while ducking the audio coming from MusicKit (application music player)

the built in Siri voices are not off sufficient quality so I am using an external service to create an mp3 file and then play this back using AVAudioSession

Sample code below

the problem I am having is that .duckOthers is not ducking the Application Music Player output

Is this a bug or am I doing this wrong?

                        // Configure audio session for system-wide ducking
                        try AVAudioSession.sharedInstance().setCategory(.playback, mode: .spokenAudio, options: [.duckOthers, .mixWithOthers])
                        try AVAudioSession.sharedInstance().setActive(true)
                        
                        // Set the ducking level to maximum
                        try AVAudioSession.sharedInstance().setPreferredIOBufferDuration(0.005)
                        
                        // Create and configure audio player
                        self.audioPlayer = try AVAudioPlayer(data: audioData)
                        self.audioPlayer?.delegate = self
                        self.audioPlayer?.volume = 1.0  // Ensure full volume for speech
                        self.audioPlayer?.prepareToPlay()
                        
                        // Set the audio player's settings for maximum clarity
                        self.audioPlayer?.enableRate = false
                        self.audioPlayer?.pan = 0.0  // Center the audio
                        self.audioPlayer?.play()