Project overview
building a supercomputer from idle computers for heavy tasks which take a long time to run on one computer
Big-dawgs
building a supercomputer from idle computers for heavy tasks which take a long time to run on one computer
Built because running ML models on a single laptop was painfully slow, while powerful machines around campus sat idle.
It lets users share unused computing power across campus so heavy tasks like ML training or simulations run faster.
It uses a lightweight server to coordinate peers and WebSockets for communication, with Python handling workload distribution and execution.
arijit built the backend architecture, including peer registration, task handling, and communication logic using FastAPI and WebSockets and tushar was runniung tests simulataneously so that system remains robust under all conditions
Getting reliable peer-to-peer communication working and ensuring tasks run correctly across multiple unpredictable devices.
Successfully built a working peer-to-peer compute network and proved tasks could run faster using shared idle hardware.
Learned how to build and coordinate distributed systems, handle real-time communication, and deal with unpredictable network behavior.
Planning to improve scheduling, add authentication, and make the system scalable enough to run across an entire campus network.