iOS Keyboard Extension Limitations

When building iOS custom keyboards with voice/audio features, these are the hard limitations discovered through the PolyVoice project.

🔴 Hard Limitations (Cannot be worked around)

1. Microphone Access — DISALLOWED

• AVAudioRecorder will fail with permission error
• SFSpeechRecognizer is unavailable
• No Siri integration from keyboard context

2. Open Other Apps — BLOCKED

• UIApplication.shared.open() returns false
• URL schemes don't work (myapp://)
• ExtensionContext.open() not available

3. Memory Limit — ~50MB

• App terminated silently if exceeded
• No crash log, just disappears
• Heavy audio processing = instant death

• Record at 16kHz mono (not 44.1kHz)
• Use 32kbps bitrate max
• Immediate file cleanup after processing
• 60-second max recording hard limit

4. No Persistent Storage

• Standard UserDefaults doesn't persist
• Must use UserDefaults(suiteName: "group.com.company.app")
• Requires App Group capability in both targets

5. Network Requires "Full Access"

• User must explicitly enable: Settings → General → Keyboard → [Keyboard Name] → Allow Full Access
• Most users won't do this
• Cannot prompt or explain from keyboard UI effectively

🟡 Partial Workarounds (User friction)

The "Open App" Workaround

Attempt:

// This does NOT work
extensionContext?.open(URL(string: "myapp://record")!)

Reality: Must use UIApplication.shared.open() outside extension context, but keyboards can't call this.

The Manual Switch Pattern

• User taps button in keyboard → Shows alert: "Open PolyVoice to record?"
• User manually switches to main app (Home button, swipe, etc.)
• Main app detects active session (via App Groups / shared state)
• Main app auto-records on appear
• Auto-stops on silence (2 seconds)
• Auto-copies to clipboard
• User manually switches back to target app
• Keyboard auto-pastes on reappear

Keyboard → Tap mic → [Manual: Switch to app] → App auto-records → 
[Manual: Switch back] → Keyboard auto-pastes

Friction points:

• Two manual app switches
• Context switching breaks flow
• Users forget to return
• Clipboard may be overwritten

🟢 Alternative Architectures

Option 1: Share Extension (Better for Audio)

• Full app capabilities
• Can record audio
• Can process and return text

Option 2: Full App Mode

• User opens app
• Records dictation
• Copies result
• Switches to target app
• Pastes manually

Option 3: Siri Shortcuts Integration

• "Hey Siri, dictate with PolyVoice"
• Returns text to current app
• Fully supported by Apple

📊 Decision Matrix

🎯 Recommendation

For voice dictation/AI transcription:

• Don't build a keyboard extension — the limitations make it frustrating
• Use Share Extension — Apple-supported, full capabilities
• Or full app — simplest to build, most reliable
• Add Shortcuts — for power users who want speed

Keyboard extension works great. Just avoid audio features.

📚 References

• Apple's official docs: https://developer.apple.com/documentation/uikit/keyboards_and_input/creating_a_custom_keyboard
• Custom Keyboard Programming Guide (WWDC sessions)
• PolyVoice project learnings (~/Projects/polyvoice-keyboard/)