Alec Go
Authored Publications
Sort By
Preview abstract
Large Language Models (LLMs) have seen increasing attentions in on-device applications due to their exceptional ability in real-world tasks. However, device-end LLM often performs suboptimal due to the hardware limitation. Cascading local (on-device) weaker and server stronger LLMs presents a promising solution to this challenge. While existing research on LLM cascade primarily focuses on optimizing the performance-cost trade-off, privacy concerns remain largely unaddressed. In this work, we prioritize privacy-preserved LLM cascading while enhancing cascade efficiency. To this end, we propose a novel CoT-enhanced policy learning strategy for deferral decision-making, which accounts for both performance-cost trade-offs and privacy considerations. Extensive experiments on three benchmark datasets validate the effectiveness and superiority of our approach.
View details
Multi-path Neural Networks for On-device Multi-domain Visual Classification
Andrew Howard
Gabriel M. Bender
Grace Chu
Jeff Gilbert
Joshua Greaves
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2021), pp. 3019-3028
Preview abstract
Learning multiple domains/tasks with a single model is important for improving data efficiency and lowering inference cost for numerous vision tasks, especially on resource-constrained mobile devices. However, hand-crafting a multi-domain/task model can be both tedious and challenging. This paper proposes a novel approach to automatically learn a multi-path network for multi-domain visual classification on mobile devices. The proposed multi-path network is learned from neural architecture search by applying one reinforcement learning controller for each domain to select the best path in the super-network created from a MobileNetV3-like search space. An adaptive balanced domain prioritization algorithm is proposed to balance optimizing the joint model on multiple domains simultaneously. The determined multi-path model selectively shares parameters across domains in shared nodes while keeping domain-specific parameters within non-shared nodes in individual domain paths. This approach effectively reduces the total number of parameters and FLOPS, encouraging positive knowledge transfer while mitigating negative interference across domains. Extensive evaluations on the Visual Decathlon dataset demonstrate that the proposed multi-path model achieves state-of-the-art performance in terms of accuracy, model size, and FLOPS against other approaches using MobileNetV3-like architectures. Furthermore, the proposed method improves average accuracy over learning single-domain models individually, and reduces the total number of parameters and FLOPS by 78% and 32% respectively, compared to the approach that simply bundles single-domain models for multi-domain learning.
View details
