Visual Reasoning for Construction Safety Monitoring

초록

The construction industry continues to report high accident rates, with scaffold-related tasks posing significant risks. While computer vision-based object detection has been widely applied to automate hazard identification, conventional approaches remain limited to recognizing isolated objects, lacking holistic reasoning over complex site contexts. Visual Question Answering (VQA) offers a promising alternative by translating explicit safety rules into natural language questions that vision–language models (VLMs) can answer, potentially enabling scene-level compliance checks. However, most existing studies treat each question independently without preserving conversational context, leaving final safety judgments reliant on manual rule aggregation. This study proposes a dialogue-oriented VQA framework for scaffold safety assessment. Seventeen OSHA-derived rules were encoded as sequential natural language questions, and a large scaffold image dataset was annotated with corresponding answers and Safe/Unsafe labels. We fine-tuned a LLaVA-1.5-7B model using LoRA to maintain contextual memory across multiple safety queries. Testing demonstrated a ~19% gain in per-question accuracy and ~15% improvement in holistic Safe/Unsafe classification over the baseline, highlighting the value of internal dialogue tracking for multi-rule assessments. These findings underscore the potential of conversational VLMs to advance automated, explainable safety monitoring in construction © 2025, Korea Institute of Construction Engineering and Management. All rights reserved.

키워드