feat: implement LLM keyword extraction and fusion scoring for table matching

- Add extract_keywords() to LLMService with conversation context support,
  using process_stream + start_log/end_log for logging consistency
- Add expand_with_terminology() to expand keywords with synonym matching
- Add fusion scoring in calc_table_embedding(): α * vec_score + (1-α) * keyword_score
- Add calc_keyword_score() with multi-level matching:
  table name exact (1.0), comment exact (0.9), comment substring (0.7),
  table name partial (0.25-0.5), field comment exact (0.5),
  field comment substring (0.4), field name partial (0.3)
- Scoring considers keyword coverage rate to differentiate tables
  with different numbers of matched keywords
- Extracted keywords persisted to ChatRecord for reuse across endpoints
- get_terminology_template() supports comma-separated keyword input
  with per-keyword matching for higher precision
- Add TABLE_EMBEDDING_ALPHA and TABLE_EMBEDDING_KEYWORD_ENABLED config
This commit is contained in:
ulleo
2026-09-14 17:51:45 +08:00
committed by ulleo
parent aac200aef5
commit 3fd6fa867d
15 changed files with 1200 additions and 29 deletions
+1
View File
@@ -809,6 +809,7 @@
"GENERATE_DYNAMIC_SQL": "Generate Dynamic SQL",
"CHOOSE_TABLE": "Match Data Table (Schema)",
"FILTER_TERMS": "Match Terms",
"EXTRACT_KEYWORDS": "Extract Keywords",
"FILTER_SQL_EXAMPLE": "Match SQL Examples",
"FILTER_CUSTOM_PROMPT": "Match Custom Prompts",
"EXECUTE_SQL": "Execute SQL",
+1
View File
@@ -809,6 +809,7 @@
"GENERATE_DYNAMIC_SQL": "동적 SQL 생성",
"CHOOSE_TABLE": "데이터 테이블 매칭 (스키마)",
"FILTER_TERMS": "용어 매칭",
"EXTRACT_KEYWORDS": "키워드 추출",
"FILTER_SQL_EXAMPLE": "SQL 예시 매칭",
"FILTER_CUSTOM_PROMPT": "사용자 정의 프롬프트 매칭",
"EXECUTE_SQL": "SQL 실행",
+1
View File
@@ -809,6 +809,7 @@
"GENERATE_DYNAMIC_SQL": "生成动态 SQL",
"CHOOSE_TABLE": "匹配数据表 (Schema)",
"FILTER_TERMS": "匹配术语",
"EXTRACT_KEYWORDS": "提取关键词",
"FILTER_SQL_EXAMPLE": "匹配 SQL 示例",
"FILTER_CUSTOM_PROMPT": "匹配自定义提示词",
"EXECUTE_SQL": "执行 SQL",
+1
View File
@@ -809,6 +809,7 @@
"GENERATE_DYNAMIC_SQL": "產生動態 SQL",
"CHOOSE_TABLE": "匹配資料表 (Schema)",
"FILTER_TERMS": "匹配術語",
"EXTRACT_KEYWORDS": "提取關鍵詞",
"FILTER_SQL_EXAMPLE": "匹配 SQL 範例",
"FILTER_CUSTOM_PROMPT": "匹配自訂提示詞",
"EXECUTE_SQL": "執行 SQL",