Hello Team,
I'm currently working on a proof of concept using Apple's Foundation Model for a RAG-based chat system on my MacBook Pro with the M1 Max chip.
Environment details:
macOS: 26.0 Beta
Xcode: 26.0 beta 2 (17A5241o)
Target platform: iPad (as the iPhone simulator does not support Foundation models)
While testing, even with very small input prompts to the LLM, I intermittently encounter the following error:
InferenceError::inference-Failed::Failed to run inference: Context length of 4096 was exceeded during singleExtend.
Has anyone else experienced this issue? Are there known limitations or workarounds for context length handling in this setup?
Any insights would be appreciated.
Thank you!
Topic:
Machine Learning & AI
SubTopic:
Foundation Models