Commit Graph
228 Commits
Author SHA1 Message Date
ulleo bc3403b8ff feat: implement LLM keyword extraction and fusion scoring for table matching
- add log
2026-09-15 10:49:12 +08:00
ulleo e37105316b feat: implement LLM keyword extraction and fusion scoring for table matching
- add log
2026-09-15 10:20:38 +08:00
ulleo e685a749c6 feat: implement LLM keyword extraction and fusion scoring for table matching
- improve table matching
2026-09-14 18:52:15 +08:00
ulleo 3fd6fa867d feat: implement LLM keyword extraction and fusion scoring for table matching
- Add extract_keywords() to LLMService with conversation context support,
  using process_stream + start_log/end_log for logging consistency
- Add expand_with_terminology() to expand keywords with synonym matching
- Add fusion scoring in calc_table_embedding(): α * vec_score + (1-α) * keyword_score
- Add calc_keyword_score() with multi-level matching:
  table name exact (1.0), comment exact (0.9), comment substring (0.7),
  table name partial (0.25-0.5), field comment exact (0.5),
  field comment substring (0.4), field name partial (0.3)
- Scoring considers keyword coverage rate to differentiate tables
  with different numbers of matched keywords
- Extracted keywords persisted to ChatRecord for reuse across endpoints
- get_terminology_template() supports comma-separated keyword input
  with per-keyword matching for higher precision
- Add TABLE_EMBEDDING_ALPHA and TABLE_EMBEDDING_KEYWORD_ENABLED config
2026-09-14 17:51:45 +08:00
junjun 8424dcbf92 refactor: Optimizing Excel header fields with too long prompts #1364 2026-09-14 10:40:52 +08:00
wangjiahao 1bc83867e5 fix: Cross-Workspace Recommended-Problem Read and Overwrite (IDOR) 2026-09-09 16:26:39 +08:00
junjun c1d333e62d fix: Cross-Workspace Data-Source Schema Disclosure and Modification (Incomplete IDOR Remediation) 2026-08-31 11:09:23 +08:00
junjun c8a2dd89bb feat: system param support key-value #1310 2026-08-10 14:09:02 +08:00
junjun 959285cfd0 feat: system param support key-value #1310 2026-08-07 11:31:28 +08:00
junjun fe65f3e2cc refactor: Optimize connection pool #1256 2026-07-20 17:32:54 +08:00
junjun 1e130223a3 refactor: Optimize the type of permission field 2026-07-16 17:09:08 +08:00
junjun 767d665dd9 refactor: Optimize connection pool 2026-07-13 17:04:52 +08:00
ulleo 2fc646ce3c feat: Adjust the organizational attribution logic of dialogues in MCP
#1221
2026-06-29 16:00:42 +08:00
junjun 1c8b6f95ae fix: Fix the vulnerability in viewing enumeration values across workspaces 2026-06-17 17:56:41 +08:00
junjun df44e660bd fix: Fix the vulnerability in viewing enumeration values across workspaces 2026-06-17 17:13:50 +08:00
Sebastion dd9627b682 fix: escape SQL values in row permission filter to prevent injection (CWE-89)
All user-controlled values interpolated into SQL WHERE clause fragments
in transTreeItem() and getSysVariableValue() are now passed through
_escape_sql_value() which doubles single quotes (standard SQL escaping)
and escapes backslashes. The logic operator in transTreeToWhere() is
also validated against a whitelist.

Affected injection vectors:
- item["enum_value"] list (enum filter)
- item["value"] string (normal filter)
- user variable values (variable filter)
- current_user name/account/email (system variable filter)
- tree["logic"] operator (tree structure)
2026-06-11 11:21:37 +08:00
junjun 5d5a92a526 fix: fix Excel sheet name inject SQL 2026-06-08 11:17:02 +08:00
junjun 348346d97b fix: remove sqlite datasource 2026-06-04 11:40:31 +08:00
junjun 76b5176b0e fix: Fix SQL injection vulnerability 2026-06-03 12:18:49 +08:00
junjun 4bfd0f49e8 fix: Fix parseExcel vulnerability 2026-06-01 12:30:51 +08:00
junjun 92c5aea70b fix: Fix parseExcel vulnerability 2026-06-01 12:28:20 +08:00
ulleo e72c0bedda fix: Fix the data source-related issues in the PR 2026-05-11 15:07:15 +08:00
junjun 37b4eb65eb fix: Fix the data source-related issues in the PR 2026-05-08 12:45:11 +08:00
yym eb587fce7b fix(Hive): normalize identifier quoting and stabilize metadata/query paths
Use Hive-compatible identifier handling and template defaults so generated SQL returns real column values, while also fixing table schema parsing and adding required Hive runtime dependencies.

Made-with: Cursor
2026-05-08 11:53:08 +08:00
yymandClaude Opus 4.6 348fd3850f feat: Add SQLite and Hive database support with sample data for AI
- Add SQLite support to data sources (file-based connection)
- Add Hive support to data sources
- Add SSL toggle option for MySQL and Doris databases
- Add sample data (3 rows JSON format) to help AI better understand table schema
- Fix SQLite schema table name issue (empty schema prefix)
- Add SQLite.yaml template for SQL generation
- Add Hive to haveSchema list in frontend

Co-Authored-By: Claude Opus 4.6 <[email protected]>
2026-05-08 11:53:08 +08:00
junjun b2de038e7a fix: Privilege Escalation Vulnerability 2026-04-30 09:49:24 +08:00
junjun 0f3529dc75 fix: Privilege Escalation Vulnerability 2026-04-23 11:38:11 +08:00
junjun 34e13a0241 fix: Excel field name is a pure numeric error message 2026-04-21 16:20:59 +08:00
fit2cloud-chenyw 4619dd4813 fix: Failed to save datasource while using Redis cache 2026-04-20 16:03:50 +08:00
junjun 0e0b8736fe refactor: upload excel error tips 2026-04-20 13:29:17 +08:00
junjun 20632f8486 fix: The system reports an error when uploading an Excel file with null values 2026-04-20 10:17:57 +08:00
junjun 7f543cf194 feat: SQLBot supports custom data types in data sources #768 2026-04-03 16:22:37 +08:00
junjun 11d1affa29 feat: MCP supports selecting workspaces #926 2026-03-24 15:40:56 +08:00
junjun e5e9e535ea fix: MySQL adds SSL configuration 2026-03-24 10:17:49 +08:00
junjun b9594285e8 feat: support system variable 2026-03-17 10:57:02 +08:00
junjun 527b18fa82 feat: support system variable 2026-03-17 10:36:05 +08:00
junjun 606949af67 feat: support system variable 2026-03-16 18:20:33 +08:00
junjun 15c75365ae refactor: The data table supports enabling and disabling 2026-03-13 14:06:25 +08:00
anch0r00 94ca094467 style: fix import order for psycopg2 2026-03-11 15:44:55 +08:00
anch0r00 39f2203cec fix: prevent SQL injection in uploadExcel via psycopg2.sql.Identifier 2026-03-11 15:44:55 +08:00
junjun 5105f8b51c feat: support system variable 2026-02-05 10:37:55 +08:00
junjun d9129073b9 feat: support system variable 2026-02-05 10:15:46 +08:00
junjun 4135aa3b1b refactor: Optimize row permission logic 2026-02-04 17:59:50 +08:00
junjun 19c38321c0 refactor: sql server support selecting compatibility with lower versions 2026-02-02 14:07:06 +08:00
fit2cloud-chenyw bbadc49759 perf: Cache clearance error in datasource deletion API 2026-01-29 10:55:53 +08:00
fit2cloud-chenyw d640ac31d1 perf: Optimize datasource-related API permission validation 2026-01-28 17:03:58 +08:00
fit2cloud-chenyw 3167783ed4 perf: Create basic assistant permissions error 2026-01-27 14:25:00 +08:00
ulleo 0dc289956b feat: improve chart field name matching with original table structure 2026-01-22 14:30:58 +08:00
wangjiahao 95b058a2ad refactor: add log 2025-12-30 17:39:55 +08:00
fit2cloud-chenyw e791eedd38 fix: API security weakness 2025-12-30 14:41:09 +08:00