Engineer • Maker • Designer
DESKTRACE
I wanted to see if I could make my desk a little more useful while working on projects. Instead of constantly switching between physical objects and information on a screen, I wondered if I could use a camera to see what was on the desk and a projector to display information directly around it. That was the starting point for DeskTrace.
Preliminary Test
The setup is fairly simple. An overhead camera looks down at the desk, while a projector points at the same area. Both connect to my laptop, which processes the camera image and sends graphics back to the projector. I used a cheap projector from Temu to get started, mainly because I wanted to see how much I could do without spending much on hardware.
Getting things to line up
The first important part was getting the camera and projector to agree on where things are. They see the desk from different positions, so simply outlining something in the camera image doesn't mean it will land in the correct position when projected. I had to calibrate the two views so that the software could map what the camera sees to the physical desk.
From there, I started working on object detection and contour tracing. The software uses computer vision to separate objects from the background, follow their edges and project the outlines back onto the desk. I've also been experimenting with contrast detection to get cleaner contours. One interesting problem is that the camera can see the projector's graphics too. If I'm projecting lines around an object, those same lines can interfere with what the camera is trying to detect. At the moment, the software briefly clears the projection to capture a cleaner image before updating the outlines. It works, although the updates aren't perfectly smooth yet.
What works so far
DeskTrace can currently detect and trace objects on the desk, display approximate measurements and project labels, grids and other reference graphics. I've also been testing object recognition and counting using objects I've taught the software to identify. The measurements are useful as quick references, but I'm not treating them as replacements for calipers. Camera position, object height and calibration all affect the result, so there's still quite a bit of work to do on accuracy.
Trying different cameras
I initially used my DJI Osmo Pocket 3 as a webcam, but I wanted to see how cheaply I could get the same basic idea working. I've since been experimenting with different cameras, including an ESP32-CAM. I even made a little transparent PLA case for it. So far, it seems I can get useful tracing from several different cameras. A better camera can provide a cleaner image and make detection easier, but it's interesting to see how much is possible with inexpensive hardware. The laptop is still doing the image processing, so the camera itself doesn't need to handle all of that.
What it costs
A few people have asked how much the setup costs. I'm using a cheap projector from Temu, and I've found that you don't need an expensive camera to get started either. I'd estimate that a similar setup could be put together for around R700 – R1,700 (roughly USD $40 – $100), excluding the computer running the software. There's also room to improve things with better hardware, but I wanted to see how far I could get with a fairly inexpensive setup.
What I'd change
There's still plenty to improve. Lighting, shadows, and objects above the desk surface can affect outlines and measurements. Moving objects quickly or having several close together can also make detection less reliable. I'd like to experiment with a better projector and camera to see how much the image quality improves. A sharper projection would make the outlines easier to see, while a better camera could help with detection. But a lot of the important improvements will probably come from calibration and software rather than simply buying more expensive hardware.
For now, DeskTrace is still an experiment. What I like about it is that I can keep changing different parts of the setup and seeing what works. It's already doing more than I originally expected from a cheap projector and a camera, and there are quite a few things I still want to try.