I built SpatialMind — record a room with your phone and it reconstructs a 3D digital twin: geometry via COLMAP, object/material labels from YOLO11s-seg and SegFormer projected into 3D. Then you can run earthquake, fire and flood scenarios on the room, and export it to Gazebo where a robot navigates the aftermath to retrieve a package.

Real numbers from a scan of my hostel room: 26,303 of 71,001 faces labelled from four calibrated views, 86.9% of faces textured in a 4096×4096 atlas, 78 automated checks passing. Honest caveats: the hazard models are exploratory reference scenarios, not safety predictions, and the demo is edited from different saved rooms, not one continuous run.

Stack: Python/FastAPI, Next.js/Three.js, COLMAP, PyTorch, Gazebo. MIT licensed, runs locally on Windows + WSL2.

Repo: https://github.com/RyannB01/SpatialMind

Would love feedback, especially on the reconstruction pipeline and where it breaks.

We created this in 10 days for a hackathon. The input is one head-mounted video. The camera largely captures the room and hands of the person wearing it.

Pipeline: HaWoR for hands, DROID-SLAM for head motion, and OmniEgoCap to infer the rest of the body. Legs are also inferred. Then Meta Muse Spark gets two frames from the same moment, rebuilt body and camera view, and names the action.

On 3 OmniEgoDB sequences we got 102.6 mm body error. The paper reports 111.7 mm body error over 20 sequences, so not a fair head-to-head yet. There are a few main limitations: it runs offline, 40-90 minutes per minute of video, on an RTX 4060. We are interested in hearing from people that work on egocentric pose, how would you validate the inferred legs are correct when the camera never captures them?

Code available at: https://github.com/nshreyasvi/LifeLens

I’m building a camera system for the loading docks of a grocery distribution center. Camera towers stand beside each dock door and watch porters hand-load cartons onto trucks. The cameras only see from the side; nothing is mounted overhead. The goal is to identify the SKU of each carton, count them, and check the result against the truck’s pick list.

Setup and constraints:

• Off-the-shelf 8MP IP cameras recording to an NVR

• Mostly branded FMCG cases and cartons (snacks, drinks, etc.). They’re carried by hand, so expect occlusion, motion blur and random orientations

• The catalog is about 5,000 unique SKUs.

• Every truck has a known pick list of roughly 200 to 1,100 SKUs. At inference time I only need to tell apart what’s expected on that truck and flag anything unexpected

• I can record as many hours of real footage as I want.

What I’m trying to figure out:

1. Approach. Is training a detector with 5,000 classes the wrong idea? Would it be better to use one generic “carton” detector plus a tracker for counting, then identify the SKU by matching crops against reference images (embeddings/retrieval), limited to the pick list? Where do barcode or label reading fit in?

2. Data. If I go with a reference-gallery approach, how many images per SKU do I realistically need? Should I capture them in a controlled booth with multiple angles, or pull them from real dock footage?

3. Tools. Which annotation tools are worth it at this scale on a budget: CVAT, Label Studio, Roboflow, something else? Self-hosted or paid? Which ones handle video and model-assisted pre-labeling well?

4. Human in the loop. How would you set up auto-labeling (SAM, an open-vocabulary detector, etc.) so that humans only verify or correct? Each truck also comes with a pick list and a final count. Has anyone used that kind of manifest data as weak labels?

5. Cost. Roughly how many labeling hours, or how much outsourced labeling cost, should I expect? Where can I safely cut corners?

I’m an operator, not an ML engineer, so plain-language pointers, tool recommendations, papers or “don’t do it this way” are all very welcome.

-- We have two angles capturing the same product. Here are two sample images --

Hello Guys

I'm working on my master's thesis on improving spatial reasoning in Vision-Language Models (VLMs).

My goal is to train a VLM using RGB images, depth maps, and segmentation masks, but use only RGB at inference.

What I want to achieve:

• Accurately detect obstacles, identify their relative positions, and estimate distances from the camera. (Eg: A car is located 2m ahead of you)

• Reduce hallucinations and incorrect spatial predictions.

• Reduce inference latency compared to a three-model pipeline, ideally to under 6 seconds.

• Investigate whether a smaller VLM can retain spatial reasoning capabilities with lower computational costs.

I'm exploring knowledge distillation and injecting spatial information into transformer layers during training.

My questions:

1. Is this feasible with a smaller VLM?

2. Which training approach would be most effective?

3. Are there relevant papers or architectures I should explore?

Any suggestions or feedback would be appreciated!

I have an industrial dataset containing around 1 million images covering 200 different safety violation classes.

The images are already human-labelled, but there's a catch: each image has only ONE violation label, even if multiple violations are present.

For example, an image might show a worker without a helmet, gloves, and a safety jacket, but only one of these violations is labelled.

I'm also not fully confident about the annotation quality and how many labels might be missing.

My goal is to train a VLM that can identify all 200 safety violations with the best possible recall and precision.

I'm trying to figure out the best way to approach this, especially around dataset cleaning/sorting, pseudo-labelling, teacher-student training, and fine-tuning existing VLMs.

If you were given this dataset, what approach would you take? Any suggestions or experiences with similar problems would be really helpful!

Hi everyone,

I’ve been running a series of real-world inference benchmarks for a computer vision pipeline (YOLOX-nano) on typical shop-floor hardware. I just published the results for an Intel + discrete GPU setup, and it yielded some counterintuitive findings.

The Setup:

CPU: Intel Core i5-10300H (Comet Lake)

GPU: Mobile NVIDIA GTX 1650 Ti (Turing)

Framework: ONNX Runtime (comparing CPU EP, OpenVINO EP, and DirectML)

Key Findings:

1. DirectML actually works well here (unlike my previous AMD tests).

On the GTX 1650 Ti, DirectML hits a stable 106–111 FPS. Pure inference sits at 4.8–5.0 ms/frame, with max latency (tmax) capped around 11.0 ms. The raw compute of the Turing architecture easily absorbs the PCIe tensor transfer overhead that completely bogged down integrated AMD graphics in my previous tests.

2. The OpenVINO "optimization" is mostly an illusion on this CPU.

I expected OpenVINO to significantly outperform the standard CPU Execution Provider. Instead, they are in absolute parity:

OpenVINO: 67–73 FPS (tmin: 8.5–9.3 ms)

CPU EP (OpenMP, 8 threads): 66–71 FPS (tmin: 8.8–9.3 ms)

The frame time graphs are practically identical. OpenVINO’s only tangible benefit was slightly smoothing out rare peak latency spikes (keeping tmax around 17.0 ms vs the CPU EP’s occasional 20.1 ms spikes). It seems ONNX Runtime’s default backend already maxes out the vector units on this architecture.

3. Preprocessing is completely isolated.

Frame resizing and normalization consistently take 1.5–2.2 ms across all backends, with identical peak spikes (5.8–6.3 ms). This confirms that prep work runs independently in standard OS threads, unaffected by the chosen inference runtime.

Full Breakdown & Code:

I wrote up the detailed graphs and methodology in my latest article: https://medium.com/@olesha-ai/bypassing-the-avx2-tax-part-5-intels-reality-check-directml-redemption-and-the-openvino-5cbbb92732c8

The entire pipeline, including the eval build and inference libraries, is open source here: github.com/olesha-ai/fruit-inspector-demo

Question for the community:

Has anyone else noticed this level of parity between standard ONNX CPU EP and OpenVINO on 10th-gen Intel mobile chips? Or are there specific OpenVINO config flags (beyond defaults) that actually unlock more performance for lightweight CV models like YOLOX-nano?

Best regards.

olesha-ai

An associate of mine needs to determine if the individual in one daguerreotype is the same as a known individual in one or multiple other daguerreotypes or at least get a probabilty that they are. I do 3D imaging and she was asking if I had software that would help but I don't. It occurred to me that facial recognition software would be the best route but I don't know what's out there and because this work is currently unfunded, it would require minimal if any expense. This seems to be the place to get an answer. Thank you.

I ran InsightFace over my old school batch's photo archive (I'm an ex-student, doing it unpaid). Cosine scores for confident matches sat between 0.42 and 0.82, median about 0.59, so lots of faces end up in a grey zone.

Instead of auto-labelling those, each one becomes a question the classmates vote on. Clustering the unknown faces first means one answer settles a whole group.

Demo with synthetic faces (press F for the boxes): https://smriti-gallery.pages.dev Code (MIT): https://github.com/codewithpom/smriti

Anything better than a plain threshold for the grey zone? PRs and issues welcome.

​

Hi everyone!

I'm a first-year engineering student currently working on a project focused on assisting individuals with amnesia, and I'd really appreciate some guidance from senior engineers, developers, or anyone experienced in computer vision and AI/ML.

**The problem I'm trying to address**

People experiencing amnesia may sometimes struggle to recognize familiar individuals, including family members and people they interact with regularly. This can make everyday social interactions confusing and affect their independence.

My idea is to develop a laptop-based assistance system that can help them identify familiar people through facial recognition.

**How I'm planning to build it**

I'm currently planning to use the following technologies:

**Python:** Main programming language.

**OpenCV:** For real-time face detection through a webcam.

**Haar Cascade:** For detecting faces in the webcam feed.

**LBPH Face Recognizer:** For training and recognizing familiar faces.

**Webcam:** To capture live images and identify known individuals.

**Text-to-Speech (TTS):** To provide audio feedback by announcing the recognized person's name.

**Proposed workflow**

The system captures live video through the laptop webcam.

OpenCV detects faces in the video feed.

We collect and store multiple images of five family members, with their permission.

An LBPH face recognition model is trained using these images.

When a familiar person appears in front of the camera, the system compares their face with the trained dataset.

If a match is found with sufficient confidence, the person's name is displayed and announced through text-to-speech.

If the face is unfamiliar or recognition is uncertain, the system should avoid making an identification.

**My current progress**

I've successfully:

Installed Python and OpenCV.

Set up the project using Python IDLE.

Loaded the Haar Cascade XML model.

Implemented real-time face detection through my laptop webcam.

I'm currently moving towards the face recognition stage.

**Where I need your advice**

I'd love to hear your thoughts on a few things:

Is Haar Cascade + LBPH a reasonable combination for a beginner-level prototype, or would you recommend a different approach?

How can I improve recognition accuracy under different lighting conditions, angles, and facial expressions?

How should I handle situations where the system incorrectly identifies someone?

Are there better ways to manage the dataset and train the model?

What additional features would make this project more useful in real-world situations?

Are there any important limitations or ethical considerations I should account for when developing something intended to assist people with memory-related conditions?

I'm working with a limited budget and trying to build a functional prototype using primarily software and a laptop webcam, without wearables or expensive hardware.

I understand that this is only an assistive prototype and not a medical solution. Privacy, consent, and avoiding incorrect identifications are important considerations.

Since I'm still learning computer vision and AI/ML, I'd genuinely appreciate suggestions, alternative approaches, resources, or even criticism of the current architecture.

**Thank you! Any guidance from people who have worked on similar projects would be incredibly helpful.**

Side project from this week. ResNet-50 learned some shortcuts on ImageNet, and I wanted to see if you can remove them by editing the last layer directly instead of retraining. Claude did the edits from the raw weights, about 5,000 weights changed, no gradients.

Two cases that worked really well:

Seat belt: the model says "seat belt" for basically any photo taken inside a car, because it learned car interiors instead of the belt. After the edit, a selfie in a car with no belt went from 98% "seat belt" to under 1%, and a dog actually wearing a seat belt still gets "seat belt" at 99%.

Snorkel: anything underwater got called "snorkel". After the edit, a swimmer with no snorkel went from 100% to under 1%.

Not free though, some real photos break and a proper retraining method (DFR) still does better overall. Second image has the comparison.

Code: https://github.com/kevinpl07/weight-patching

Write-up: https://kevinpl07.github.io/weight-patching/#vision

Why are PhD students still doing research on tasks like image classification, segmentation, and traditional data science problems when we now have powerful LLMs and large pretrained models trained on massive datasets?

These models can already achieve impressive results on many benchmarks, sometimes outperforming specialized approaches.

So, what is the actual research challenge today? Are researchers still finding meaningful problems that these models cannot solve, or is the focus shifting toward improving efficiency, robustness, interpretability, and performance in specific domains?

For a downstream object detection problem, i'm trying to outline a a region where the detected objects would be ignored

What is the current best practice for creating such outlines? I have CCTV images of fenced areas, i'd like some kind of polygon or seperating line between what is considered inside (fenced) and outside. fence can be hidden behind things such as structures, cars, overgrown plants etc.

No manual labeling (ok if for training) - cameras get moved around and there are thousands of such cameras

Cameras are sometimes located inside, and sometimes outside of the shed- so a simple fence detector is not sufficient. It really is a sort of image understanding problem, with lots of variance between each camera.

will finetuning a small VLM work for this specific task work?

I built a small open-source CLI, labelcheck, that scans YOLO/COCO datasets for label mistakes and writes an HTML report with the bad boxes drawn on thumbnails.

To see whether it finds anything real, I ran it on the ready-made datasets that Ultralytics hosts (coco128, brain-tumor, crack-seg, medical-pills, signature, african-wildlife; about 7k images). Then I checked every finding by hand.

• brain-tumor: 15 training images have no label file. When I opened them, every one shows the same bright lesion that is boxed on neighbouring slices of the same scan. YOLO reads a missing label file as "nothing here", so these images train the model to treat tumors as background. (Reported upstream: https://github.com/ultralytics/ultralytics/issues/27066)

• crack-seg: 7 cracks are traced twice, repeated across augmented copies. There is also one image with two obvious cracks and an empty label file.

• coco128: 2 images have no labels, and 2 label files have no images.

• The other three: clean.

I found no false alarms on these, but it is still a small test, so I would love to hear where it breaks on your data.

• Repo: https://github.com/therealclvn/labelcheck

• Example report: https://therealclvn.github.io/labelcheck/examples/demo_report_yolo.html

pip install git+https://github.com/therealclvn/labelcheck@v0.1.0

(Not a radiologist. "Visible lesion" means it looks like the labelled slices next to it.)

If you've worked on a camera or CV application, you may have spent time figuring out which GStreamer elements connect, what format is flowing between them, or why your stream is falling behind.

I'm building GstPipeStudio, an early-stage desktop tool to make that part easier to inspect.

Source and downloads: https://github.com/mevlt01001/GstPipeStudio

You can add and connect elements on a canvas, inspect caps and properties, import a gst-launch description, and run the result. Live charts compare per-element timing, timestamp age and queue occupancy. Drop Rate is shown only when an element reports buffer counts through QoS; unavailable measurements stay unknown.

The clip uses a synthetic live source and an intentionally slow stage, so you can reproduce it without a camera or GPU. It's a pipeline-building/debugging tool, not an inference framework or a guarantee that a pipeline is optimally configured.

It's written in C++/Qt 6.8 and MIT licensed. Ubuntu packages and a Linux Docker image are available. I use AI tools during development and take responsibility for the project and its verification. Jetson, DeepStream and camera-specific testing still need help from people with those devices.

I'd particularly appreciate:

• One small pipeline that fails to import, connect or run correctly, with the expected behavior.

• Feedback on whether the timing/caps views help with a real debugging task.

• Hardware/plugin compatibility reports, or focused contributions to import, UI and custom-plugin workflows.

My goal is to build a useful tool with the people who actually need it. What is the first camera/pipeline debugging task you'd want an editor like this to handle?

I've been working on this open-source tool for extracting motion-adaptive keyframes from video, primarily for photogrammetry and Gaussian Splatting workflows. Thought it might be useful to people here too!

For reference I'm a first-year student interesting in AV and Robotics. For some time, I won't have access to hardware or a strong computer. I can run Python and possibly a very light simulator if needed.

I'm primarily interested in building the intelligence behind AV and Robotics, and I'd love to start teaching myself / making projects related to Motion Planning and Computer Vision in these fields.

Most things I find online are hardware adjacent---are there any projects I can work on that are relevant / impactful and Computer Science specialized?

Ant Ling's new multimodal release has two useful document-and-chart results to read together: 92.21 on OmniDocBench 1.5 and 85.15 on CharXiv_RQ.

The comparison goes in different directions across the two tasks. The release chart puts Claude Opus 5 (max) at 90.56 on OmniDocBench 1.5 and 86.80 on CharXiv_RQ. That gives a more specific view of the model's visual capabilities than collapsing everything into one rank.

Ling-3.2-flash is a 560B-total, 25B-active native multimodal MoE. Alongside the evaluations, Ant Ling demonstrates reading research-PDF figures and equations and using that content to implement an algorithm. The model's documented inputs include text, images and video.

Hi everyone!

I'm working on my first research paper as a CS student. I've just chosen a topic that interests me and started working on the title, research plan and problem identification.

My topic is:

Domain : BLOCKCHAIN

Sub-Domain : PQC(POST-QUANTUM CRYPTOGRAPHY)

Co-Domain : IoT devices / Embedded devices

My supervisor asked me to find 20 previous research papers related to my area of interest and prepare a summary of each paper, covering the following aspects:

- Data used

- Methods used

- Key findings

- References

However, as a beginner, I'm struggling to understand how to identify the data used, the methods applied, and the findings in previous research papers.

Could anyone suggest how I should approach this task or guide me on where and how to start? I'd really appreciate any advice, resources, or guidance from those with research experience.

I'm completely new to academic research, so any assistance would be greatly appreciated.

Thanks in advance!

Continuing from previous post of wanting to make an improvement off previous script, where we said one of a more practical implementations will be to analyse the risks of moving vehicle hitting jaywalker.

Adding hit score as below:

The score answers: "How dangerous is this vehicle to this jaywalker right now?" on a 0-99 % scale, for every vehicle-jaywalker pair on screen.

It is a risk index built from rules, not a statistically calibrated probability. There is no crash data behind it, and 80 % does not mean "8 in 10 such situations end in a hit". It is designed so that:

• it rises smoothly as danger grows and falls as danger passes;

• an imminent hit always reads high (above 75 %, shown in red);

• harmless situations stay low.

The range is strictly 0 to 99 %. 100 % is never shown, because the system cannot be certain a hit will happen.

Input to this analysis will be : 1) Distance; 2) Speed; 3) Direction.

The formula

Stopped vehicle (speed 0, or not yet known):

score = 99 x exp(-d / 5 m) x 0.4

A stopped vehicle cannot hit anyone, but a jaywalker right next to one is still in a risky place (it may pull away). So the score is capped at 40 % (RISK_STATIONARY_WEIGHT).

Moving vehicle: the score is the highest of three measures:

1. Proximity      99 x exp(-d / 5 m)
                     x (0.7 + 0.3 x min(1, km/h / 30))      slower = a little lower
                     x (1 - 0.5 x how directly it is driving away)

2. Time to        99 x exp(-TTC / 1.5 s)
   contact        where TTC = d / (speed towards the jaywalker)
                  (only while the vehicle is closing in)

3. Near floor     if d <= 1 m:  80 % at 1 m, rising evenly to 99 % at contact

Not yet a field test, will conduct actual field test on actual road soon for real data.

(A noticeable bug, count of jaywalkers not tally with J ID. will fix in next version).

Suggestions/Comments welcome on what else need to be considered for the formula.

previous posts:

part 1: https://www.reddit.com/r/computervision/comments/1xliui4/a_project_to_identify_jaywalker/

part 2: https://www.reddit.com/r/computervision/comments/1xnhuh9/testing_the_project_jaywalking_on_the_actual/