Define compatibility as a supported outcome
Write the important user task and its pass condition before selecting devices. For a booking app, that might include choosing an option, entering details and receiving confirmation. A configuration passes when the intended supported outcome is usable, even if platform conventions make some controls look different. Document deliberate differences so testers do not mistake them for failures.
Choose matrix dimensions independently
Record platform, manufacturer, model, operating system, screen or window state and app build. Android quality guidance includes varied form factors and display modes, while Apple describes layouts adapting to available window space. Add phones, tablets or foldables when your product supports them. Avoid treating a screen-size preview as another installed-device configuration.
Make platform changes part of the plan
Android behavior changes can apply broadly or depend on the target SDK, so operating-system testing should be tied to the APIs and journeys your app uses. For either platform, review the relevant release documentation and keep the app build constant when comparing environments. Add a reported failure configuration even when it is uncommon, because it answers a specific support question.
Keep comparison runs consistent and manageable
Use the same test account state, data and expected result across the shortlist. Separate setup failures from product failures: an installation problem needs different evidence from a rejected form. Prioritize essential journeys first, then explore permission and recovery states. Review the matrix after a release or a change in audience use, removing entries whose purpose no longer applies.
Your testing checklist
- Document supported platforms and OS ranges
- Give every selected configuration a reason
- Use the same build and controlled starting state
- Compare one complete critical journey across the matrix
- Add relevant permissions, interruptions and recovery
- Retest the failing environment and a comparison configuration
A practical workflow
Define the risk and support target
Use product requirements, customer use and recent issues to set the scope.
Build and confirm the matrix
Select representative configurations and request the exact coverage you need.
Compare results against one outcome
Record differences, prioritize their impact and verify fixes with consistent steps.
Common questions
There is no universal number. Begin with configurations that represent your supported customers and important risks, then expand when usage or findings reveal a gap.
The important requirement is a usable supported outcome. Document platform-specific design decisions and compare behavior, accessibility and task completion rather than requiring identical pixels.
No. A pass describes the build, configurations and scenarios actually reviewed. Keep the tested scope visible and update it as your product and customer usage change.
Technical references
Official documentation for the underlying testing concepts. Available capabilities vary by provider and setup.
Android Developers: compatibility framework toolsAndroid Developers: adaptive app quality guidelinesApple Human Interface Guidelines: layout