Research
Semantic complexity estimation and price-based multi-agent negotiation reduce tail latency and energy in edge-cloud generative inference
Nature ·
The Brief
- The paper proposes estimating the semantic complexity of requests to guide generative inference across edge devices and the cloud.
- It adds price-based multi-agent negotiation to allocate inference work between edge and cloud.
- The authors report that these methods reduce tail latency and energy use in edge-cloud generative inference.
Why it matters
Tail latency and energy are practical limits on running generative AI models outside large data centres, so a scheduling approach that addresses both could inform how edge-cloud systems are built.
Who’s affected
- researchers and engineers working on edge-cloud AI inference
- system designers concerned with latency and energy efficiency of generative models
- #edge computing
- #generative ai
- #inference
- #latency
- #energy efficiency
- #multi-agent systems
Timeline
Article appeared in the Nature Scientific Reports subject feed
Physics : nature.com subject feeds
Semantic complexity estimation and price-based multi-agent negotiation reduce tail latency and energy in edge-cloud generative inference