The Connectivity Shift-Left initiative is tracking to Stage 1 completion on the Vega pilot by May 15 with FOS targeted for May 30. Three quality gates are defined and operational: Gate A (a/m/d pre-VML, middleware P0+P1 at 100% automated), Gate B (Two sub gates — B1: CR-merge functional BAT via Loom/CRUX blocking with a 4-hr SLA, B2: non-blocking targeting <1 day), and Gate C (vendor FW regression on the BSP pipeline).
For the pilot we selected Kanto (Vega TV), Brioche (Vega EFD), and Seabright (FOS8). Kanto and Brioche setups are ready across all gates and technology with 12 functional setups (Each Gate has 1 setup per technology — WiFi, BT and SmartHome) deployed and 50-iteration stability runs completed at >95% setup stability (last week: 20k TCs executed, ~97% setup stability across WiFi/BT).
Seabright Stage 1 is at stability validation phase — Gate definitions are same as Vega except Gate A, test case identification and infra build completed for Gate B1 and currently blocked for other gates due to constrained device availability in India.
On the tooling side, VegaLoom E2E POC is complete on one BT package and CRUX rule rollout is in progress across 70+ Connectivity packages; We will be live for 70%+ of the packages by end of this week (Non-blocking). VegaLens ground-truth and config validation are done and working on test case mapping with team for next phase (currently test running on gates has static coverage now, Phase 2 dynamic when production-ready). Completed and shared the setup requirement for all future deployment with LMS team.
🔵 This Week's Highlights (Apr 22 – May 1)
- Test setup stability continues at >95% across 12 functional setups
- VegaLoom: 70+ packages targeted, 20 onboarded; targeting 70%+ live by EoW (non-blocking)
- Working on identifying auto trigger for a/m/d on actual device instead of reference platform
- Automation log post-processing: categorizes setup, script, config, and DUT issues (demo completed)
- Setup requirement for all future deployment completed and shared with LMS team
- 50-iteration stability runs completed across all gates
🟣 Next Steps (May 1–9)
- Go OFB for shift left for Vega devices (Kanto and Brioche) — test execution as non-blocking for CRUX rule
- Continue enabling Gate B1 for FOS, start test case execution of B1 via PreCI pipeline
- Address Dev team feedback on VegaLoom improvements with Loom POC
- Complete automation log post-processing integration and truth validation across setups
- Target: 70%+ VegaLoom packages live (non-blocking)
- Begin test case mapping with VegaLens team for Phase 2 dynamic coverage
| Phase | Work Stream | Vega Status | Vega ETA | FOS Status | FOS ETA |
|---|---|---|---|---|---|
| Planning | Quality Gate Definition | ✅ Completed | Mar '26 | ✅ Completed | Mar '26 |
| Planning | Test Case Identification | ✅ 1,189 test cases | Mar '26 | ✅ Completed | Apr 17 |
| Planning | Setup Feasibility | ✅ Validated | Mar '26 | ✅ Completed | Apr 17 |
| Execution | Infrastructure Build | ✅ 12 setups, >95% | Mar '26 | 🟡 B1 complete; others blocked | Apr 24 |
| Execution | Setup Stabilization (10 iter) | ✅ Completed | Mar '26 | 🟡 Stability validation | Apr 30 |
| Execution | Pipeline Integration (Loom) | ✅ Completed | Apr 17 | Not Started | Apr 30 |
| Execution | E2E POC (10 iterations) | ✅ Completed | Apr 24 | Not Started | May 15 |
| Execution | CRUX Rule (all Conn Packages) | 🟡 In Progress | Apr 30 | — | — |
| Dev Deploy | Training & Onboarding | Not Started | Apr 30 | Not Started | May 15 |
| Dev Deploy | Gradual Rollout | Not Started | Apr 30 | Not Started | May 15 |
Data from DOTS pools. Pass Rate = Pass/(Pass+Fail). DUT Issues from BT Shift-Left Stability Quip.
| Gate | Technology | Past 7 Days | Past 30 Days | Key DUT Issues (JIRA) | ||||
|---|---|---|---|---|---|---|---|---|
| TCs | Pass | Rate | TCs | Pass | Rate | |||
| Gate A | WiFi | 1,009 | 870 | 86.2% | 1,009 | 870 | 86.2% | — |
| BT | 895 | 519 | 58.0% | 1,370 | 822 | 60.0% | CARNIVAL-459, CARNIVAL-458, ACE-131487, ACE-122168 | |
| Gate B | WiFi (B1) | 586 | 563 | 96.1% | 813 | 699 | 86.0% | — |
| BT (B2) | 3,077 | 2,436 | 79.2% | 3,937 | 2,927 | 74.3% | ACE-131786, ACE-132374, CONNQA-32751, CONNQA-33151 | |
| Gate C | BT | 630 | 566 | 89.8% | 8,262 | 7,462 | 90.3% | ACE-131786, ACE-132374, PROJVEGA-317842 |
| TOTAL | 6,197 | 4,954 | 79.9% | 15,391 | 12,780 | 83.0% | — | |
| Attribute | Gate A | Gate B | Gate C | |||
|---|---|---|---|---|---|---|
| WiFi | BT | WiFi (B1) | WiFi (B2) | BT (B2) | BT | |
| Pool | Connectivity_WIFI_Gate_A | Connectivity_BT_Gate_A | Connectivity_WIFI_Gate_B1 | CONNQA-WiFi-GateB2 | Connectivity_BT_Gate_B2 | Connectivity_BT_Gate_C |
| TMS | WiFi Gate A TMS | N/A | WiFi Gate B1 TMS | WiFi Gate B2 TMS | BT Gate B2 TMS | BT Gate C TMS |
| Device Onboarded | Kanto, Brioche | Kanto, Brioche | Kanto, Brioche | Kanto, Brioche | Kanto, Brioche | Kanto, Brioche |
| Dashboard | WiFi Gate A | — | WiFi Gate B1 | WiFi Gate B2 | — | — |
| Run Time | — | — | — | — | Kanto: 10 hrs, Brioche: 9 hrs | Kanto: 5h 45m, Brioche: — |
| 7d Pass Rate | 86.2% | 58.0% | 96.1% | — | 79.2% | 89.8% |
Source: Sub-Zero Test Cases wiki · DUT Issues: BT Shift-Left Stability Quip
Click any status cell to update. Changes are saved locally in browser (localStorage).
| Tool | Purpose | Status | Details | Last Updated |
|---|---|---|---|---|
| Loom (CRUX) / VegaLoom | CR merge pipeline trigger for Gate B1 | In Progress | E2E POC complete on 1 BT package. 70%+ packages targeted live by EoW (non-blocking). CRUX rule rollout across 70+ Conn packages. | May 1, 2026 |
| VegaLens | Dynamic impact-based test selection | In Progress | POC done. Ground truth & config validation completed. Working on test case mapping for Phase 2. Currently static coverage; Phase 2 dynamic when production-ready. | May 1, 2026 |
| ARAM Dashboard | Org-wide shift-left readiness | Completed | Connectivity data validated. Pending: Automated Test Stability % from tool owner. | Apr 17, 2026 |
| Boomerang | Per-gate execution dashboards | Completed | A1, A2, B live. C, D in progress. | Apr 17, 2026 |
| Smoke Test | Setup sanctity before every run | In Progress | Design finalized. Validates DUT, AP, DLI, host, RasPi. Halts on P0 failure. Code merged. | Apr 25, 2026 |
| Log Post-Processing | Auto-categorize failure types | In Progress | Categorizes setup, script, config, and DUT issues. Demo completed. Integration and truth validation in progress. | May 1, 2026 |
These are real device bugs caught by automated gates before code reaches VML.
| JIRA | Tech | Impact | Description |
|---|---|---|---|
| AZMAXIII-5451 | WiFi | High | Registration fails after reboot on non-secure HVT devices (~44 TCs) |
| PROJVEGA-377465 | WiFi | High | MTK ap0 connectivity layer — HostAP blocked (22 TCs) |
| ACE-131786 | BT | Medium | Brioche EVT HW — A2DP pairing/concurrency (~14 TCs) |
| ACE-132374 | BT | Medium | Brioche HOGP pairing failures (~8 TCs) |
| CARNIVAL-459 | BT | Medium | A2DP streaming reboot/concurrency intermittent (~6 TCs) |
| PROJVEGA-382865 | WiFi | Medium | Hard reboot issue on Kanto (Gate A2) |
| # | Risk/Blocker | Severity | Owner | Mitigation/ETA | Status |
|---|---|---|---|---|---|
| 1 | Device Availability: Brioche short by 4 devices in Sunnyvale. Seabright limited in India. | High | LMS/Infra | Procurement tracked | In Progress |
| 2 | Gate C Trigger: MTK FW regression trigger in BSP pipeline not finalized | Medium | Dev Team | Address after Loom ready | Planned |
| 3 | SmartHome MTS: Blocked on KT for trigger mechanism | Medium | chenarav | Zigbee setup in progress | In Progress |
| 4 | VegaLoom access: Team needs access for most connectivity packages | Medium | gunu | Working with POCs for CRUX rule | In Progress |
Case IDs from TeMet 2025. Fields queried LIVE from TestRail (May 12, 2026). Filters: P0+P1, Functional (trail contains "functional"), Section trail exclusion.
| Technology | Product | Total TCs | Automatable | Automatable % | Automated | Automated % | Coverage | Remaining |
|---|---|---|---|---|---|---|---|---|
| Wi-Fi | Overall (dedup) | 505 | 328 | 65% | 101 | 20% | 30.8% | 227 |
| ↳ Wi-Fi | EFD | 380 | 301 | 79% | 92 | 24% | 30.6% | 209 |
| ↳ Wi-Fi | FireTV | 441 | 297 | 67% | 95 | 22% | 32.0% | 202 |
| BT | EFD | 333 | 305 | 92% | 126 | 38% | 41.3% | 179 |
| BT | FireTV | 71 | 57 | 80% | 44 | 62% | 77.2% | 13 |
| SmartHome | EFD/FTV | 366 | 277 | 76% | 110 | 30% | 39.7% | 167 |
| TOTAL | — | 1,275 | 967 | 76% | 381 | 30% | 39.4% | 586 |
Filters: P0 (priority_id=11) + P1 (priority_id=1) only · Functional (section trail contains "functional") · Exclude deleted/archived sections
Fields: custom_automatable=1 (Yes) · custom_automated=True
Wi-Fi: TeMet old IDs → mapped to suite 408234 via 2020_to_2025_id_mapping.json. Product split via custom_compatiblefordevicefamily + custom_device_affected fallback. Overall row = deduplicated unique count (used in grand total).
Note: Co-ex excluded from shift-left scope.
⛔ Excluded Features: Remote, BLE Audio, UGS, ZTS, OTA, FFS, Coex, Co-ex, Concurrency, Ginger, Alibaba, ASHA, AVLS, Aquila, Luma, Airplay, Radar, Ethernet, X-rey, Miracast, Roaming, A2A, OOBE, Eero, OTA12:18, WHA, Captive Portal, Metrics, Tethering, AP Selection — test cases with these keywords in section trail are not included in the above counts.
📂 Raw Data: Shift-Left Functional P0-P1 by Component (SharePoint Excel) — one sheet per Component × Product for POC assignment.
Tracking 3 metrics across sprints. Format: xx% (numerator/denominator). Sprint cadence: 2 weeks (Mon–Sun).
Wi-Fi (Overall)
BT - EFD
BT - FireTV
SmartHome - EFD/FTV
| Technology | Metric | Sprint-1 (Apr 27–May 10) | Sprint-2 (May 11–24) | Δ |
|---|---|---|---|---|
| Wi-Fi | Automated / Total | — | 9.0% (106/1174) | NEW |
| Automatable / Total | — | 50.3% (591/1174) | NEW | |
| Automated / Automatable | — | 17.9% (106/591) | NEW | |
| ↳ Wi-Fi EFD | Automated / Total | — | 16.8% (104/618) | NEW |
| Automatable / Total | — | 67.3% (416/618) | NEW | |
| Automated / Automatable | — | 25.0% (104/416) | NEW | |
| ↳ Wi-Fi FireTV | Automated / Total | — | 9.8% (100/1023) | NEW |
| Automatable / Total | — | 52.0% (532/1023) | NEW | |
| Automated / Automatable | — | 18.8% (100/532) | NEW | |
| BT - EFD | Automated / Total | 43.8% (176/402) | 42.8% (176/411) | ▼ -1.0% |
| Automatable / Total | 88.8% (357/402) | 89.1% (366/411) | ▲ +0.3% | |
| Automated / Automatable | 49.3% (176/357) | 48.1% (176/366) | ▼ -1.2% | |
| BT - FireTV | Automated / Total | 17.3% (91/525) | 17.3% (91/525) | — 0.0% |
| Automatable / Total | 49.0% (257/525) | 49.0% (257/525) | — 0.0% | |
| Automated / Automatable | 35.4% (91/257) | 35.4% (91/257) | — 0.0% | |
| SmartHome | Automated / Total | 25.1% (120/479) | 25.1% (120/479) | — 0.0% |
| Automatable / Total | 76.8% (368/479) | 76.8% (368/479) | — 0.0% | |
| Automated / Automatable | 32.6% (120/368) | 32.6% (120/368) | — 0.0% |
sprint_history.json.
| # | Case ID | Title | Technology | Section | Priority | Status |
|---|
Total cases in scope: 1,818. Use filters to narrow view. Data from TestRail live query (May 5, 2026).