Apple Minions cuts 10-hour AI inference tasks to minutes using idle Mac GPUs

Apple Research has published the details of their work on Minions. Minions is a solution for harnessing the power of idle Mac GPUs to create a distributed computing resource for machine learning jobs. This is interesting as it helps developers run large inference jobs in a matter of minutes, as opposed to waiting over 10 hours for an available GPU if they had a dedicated GPU. It's a way to cut down the costs of using specialized, expensive hardware by utilizing idle computing resources in the office environment.

Apple research system repurposes idle Mac GPUs to slash machine learning wait times

Minions is a virtual GPU cloud that dynamically assigns work across macOS machines. There is a central Opinion Server that helps orchestrate the jobs. Minion Executors are assigned to individual Mac machines to execute the work.

• Metadata Storage: PostgreSQL
• Monitoring Dashboard: Grafana
• Execution Model: Pull-based

There is a Minion Watchdog that helps to manage the connectivity of the machines. It helps to deal with sleep states, disconnecting, power failures, etc. by re-assigning jobs to other available machines.

PostgreSQL is used to maintain the metadata regarding the job/machine state. It uses a pull model to execute. The executors pull for the work rather than being pushed the work.

The work is monitored in real-time with a Grafana dashboard. This helps to ensure the number of machines available and their state while the work is being executed.

It's important to note that according to Apple Research, this is not a consumer-facing product for the public. This is an internal solution to be used by engineering and research teams. This is important as this helps to address data security concerns when multiple departments are using the same hardware resources. The goal is to reduce the infrastructure costs by trying to get the most out of the existing Mac hardware for AI.

Minions is a good example of how idle distributed resources can help speed up machine learning inference jobs. It's also a way to manage machine availability and assign work in a corporate environment. This technology is still in the research phase and they are exploring its usefulness for the internal development needs at Apple.