폐쇄망 환경에서의 항공·방산 기술문서 대상 BM25+FAISS 하이브리드 검색을 이용한 RAG 검색 파라미터 최적화 연구

초록

Closed-network deployments of RAG systems cannot rely on cloud-based retrieval, yet most existing work assumes unrestricted API access. We built a fully offline BM25+FAISS hybrid retrieval pipeline and tested it on 500 KIPRIS patents and 700 NASA NTRS reports (~39,900 chunks). Varying hybrid weight (α), preprocessing, top-k, and similarity threshold, we measured Recall@k, nDCG@k, and MRR across configurations. The dense-favoring setting (α=0.2) reached nDCG@5 = 0.9948 and Recall@5 = 0.905 — a 64.5% gain over standalone BM25 and 26.6% over FAISS (p<0.05) — while k=8 yielded perfect recall across all α values. This work provides a reproducible offline retrieval framework, domain-specific validation, and practical parameter guidance for security-constrained RAG deployments.

키워드