Files
new-api/model/task_model_alias.go
Calcium-Ion 128089fa7d feat(i18n): translate web console messages in the frontend and AI-client errors in the backend (#7712)
* feat(i18n): translate web console messages in the frontend and AI-client errors in the backend

Chinese text was hard-coded in API messages, stored log content, emails and
server logs, so it reached users of every interface language.

Text is now translated where its reader is:

- Web console APIs return the English source text as message_key with
  message_params, and the web console translates it in all seven languages.
  message keeps the English text for other API callers.
- Errors returned to AI clients, emails and notifications are translated by
  the backend (en, zh-CN, zh-TW) in the language saved on the account, then
  the language of the request's Accept-Language header.
- A reader who states no language still gets the original Chinese text of the
  messages that were Chinese before, word for word. Gateways that use another
  new-api as their upstream match that text, for example to disable a channel
  whose account ran out of quota.
- Usage, top-up and system log content is stored as content_parts (key and
  params) and rendered in the viewer's language. Task fail reasons and refund
  reasons written by the gateway are English keys. Rows written earlier are
  shown as before.
- Server logs are English.

Also:

- The saved interface language zhTW, and zh-HK, zh-MO and zh-Hant, now select
  Traditional Chinese in the backend instead of Simplified Chinese.
- The built-in group descriptions are translated; descriptions an
  administrator saved are shown as written.
- .agents/rules/i18n.md records the conventions. Tests in i18n/ fail on a
  non-English string literal, on a web console message without translations
  and on a locale key without a constant.

* feat(i18n): print server logs in Chinese when LOG_LANGUAGE asks for it

Server log lines are English by default. With LOG_LANGUAGE=zh-CN, the lines
that were Chinese before print their original Chinese text.

common.LogText formats a log line from its English format, which is also the
key of its Chinese text in common/log_text.zh-CN.json. A line without an entry
prints in English. go vet checks the arguments of every call, and
TestServerLogTexts checks that each entry still matches a call and has the
same verbs as its key.

* docs: list LOG_LANGUAGE in the docker compose example

* feat(i18n): translate every server log line and add DEFAULT_LANGUAGE

Every line the gateway prints goes through common.LogText. With
DEFAULT_LANGUAGE=zh-CN, the lines that have a Chinese text in
common/log_text.zh-CN.json print it and the others print English.

DEFAULT_LANGUAGE replaces LOG_LANGUAGE. It also sets the language of
backend messages, emails and notifications for a reader who states no
language: en, zh-CN or zh-TW, English when unset. The list of messages
that stayed Chinese for such readers is removed.

Review fixes:
- Channel test logs store the token name "Model test"; the console shows
  it in the interface language.
- Task fail reasons written by the gateway are fixed sentences the console
  translates. The HTTP status and the upstream detail go to the server log
  when a polled task is failed.
- The auto-disable reason of a slow channel is written in DEFAULT_LANGUAGE.

Also:
- Drop the repository-scanning lint tests, the i18n rules file and its
  read gate. AGENTS.md keeps the conventions in four lines.
- Payment error toasts keep their previous text and only add the
  translated message key.
- Remove guards and helpers that no caller needs.

* fix(i18n): address review on language lookup, notices and error responses

- Relay requests no longer load the user a second time to find the
  language: the settings TokenAuth put into the request are used as they
  are, also when they hold no language.
- Upstream model update notices are rendered in each recipient's saved
  language, DEFAULT_LANGUAGE when none.
- /api/usage/token and /api/log/token answer with backend-translated
  messages again, with the keys and status codes used before this PR.
  Their callers are scripts, not the web console.
- Webhook notifications fill {{value}} placeholders like the other
  senders. Running the rendered text through Sprintf appended
  %!(EXTRA ...) and broke %2F in links.
- RootUserLanguage reads the setting of the root user it already loaded.
- zh_TW and zh_HK are read as Traditional Chinese.
- Add common.ApiErrorStatus for error responses under another HTTP status
  and use it where those responses were built by hand.
- Remove middleware.GetLanguage, which had no callers.
2026-10-09 21:39:51 +08:00

212 lines
5.6 KiB
Go

package model
import (
"sync"
"sync/atomic"
"time"
"github.com/QuantumNous/new-api/common"
"github.com/QuantumNous/new-api/pkg/jsplugin"
)
// TaskAliasTarget is one mapping-derived alias after cross-channel aggregation.
// Declared is empty when the same plugin resolves the alias to more than one
// declared tail (display-only; pin is disabled).
type TaskAliasTarget struct {
Alias string
Declared string
PluginKey string
}
type taskAliasView struct {
generation uint64
expiresAt time.Time
byFold map[string]TaskAliasTarget
}
const taskAliasViewTTL = 60 * time.Second
var (
taskAliasViewPtr atomic.Pointer[taskAliasView]
taskAliasRebuildMu sync.Mutex
)
// ResolveTaskModelAlias returns the mapping-derived alias target for name
// (exact or ASCII-folded). g is the caller's routing generation: a Number
// mismatch or TTL expiry rebuilds the view against that generation.
func ResolveTaskModelAlias(g *jsplugin.RoutingGeneration, name string) (TaskAliasTarget, bool) {
if g == nil || name == "" {
return TaskAliasTarget{}, false
}
view := loadFreshTaskAliasView(g)
if view == nil {
return TaskAliasTarget{}, false
}
target, ok := view.byFold[jsplugin.ASCIIFold(name)]
return target, ok
}
func loadFreshTaskAliasView(g *jsplugin.RoutingGeneration) *taskAliasView {
view := taskAliasViewPtr.Load()
if taskAliasViewFresh(view, g.Number) {
return view
}
taskAliasRebuildMu.Lock()
defer taskAliasRebuildMu.Unlock()
view = taskAliasViewPtr.Load()
if taskAliasViewFresh(view, g.Number) {
return view
}
rebuilt := buildTaskAliasView(g)
taskAliasViewPtr.Store(rebuilt)
return rebuilt
}
func taskAliasViewFresh(view *taskAliasView, generation uint64) bool {
return view != nil && view.generation == generation && time.Now().Before(view.expiresAt)
}
func rebuildTaskAliasView() {
taskAliasRebuildMu.Lock()
defer taskAliasRebuildMu.Unlock()
taskAliasViewPtr.Store(buildTaskAliasView(jsplugin.DefaultRegistry.Generation()))
}
type taskAliasDraft struct {
spellings []string
byPlugin map[string]map[string]struct{}
}
func buildTaskAliasView(generation *jsplugin.RoutingGeneration) *taskAliasView {
genNum := uint64(0)
if generation != nil {
genNum = generation.Number
}
view := &taskAliasView{
generation: genNum,
expiresAt: time.Now().Add(taskAliasViewTTL),
byFold: make(map[string]TaskAliasTarget),
}
if DB == nil {
return view
}
var channels []Channel
err := DB.Select("id", "type", "models", "model_mapping").
Where("status = ?", common.ChannelStatusEnabled).
Find(&channels).Error
if err != nil {
common.SysError(common.LogText("rebuild task alias view: %s", err.Error()))
return view
}
drafts := make(map[string]*taskAliasDraft)
for i := range channels {
channel := &channels[i]
mappingJSON := channel.GetModelMapping()
if mappingJSON == "" || mappingJSON == "{}" {
continue
}
modelMap := make(map[string]string)
if err := common.UnmarshalJsonStr(mappingJSON, &modelMap); err != nil {
common.SysError(common.LogText("task alias view: channel %d model_mapping: %s", channel.Id, err.Error()))
continue
}
inModels := make(map[string]struct{})
for _, modelName := range channel.GetModels() {
inModels[modelName] = struct{}{}
}
for alias, mapped := range modelMap {
if mapped == "" {
continue
}
if _, exposed := inModels[alias]; !exposed {
continue
}
if _, declared := generation.CanonicalModel(alias); declared {
continue
}
tail, cyclic := followChannelModelMapping(modelMap, alias)
if cyclic {
common.SysError(common.LogText("task alias mapping cycle dropped: channel=%d key=%q", channel.Id, alias))
continue
}
declared, ok := generation.CanonicalModel(tail)
if !ok {
continue
}
plugin, ok := generation.GetByModel(declared)
if !ok {
continue
}
fold := jsplugin.ASCIIFold(alias)
draft := drafts[fold]
if draft == nil {
draft = &taskAliasDraft{byPlugin: make(map[string]map[string]struct{})}
drafts[fold] = draft
}
draft.spellings = append(draft.spellings, alias)
declareds := draft.byPlugin[plugin.Meta.Key]
if declareds == nil {
declareds = make(map[string]struct{})
draft.byPlugin[plugin.Meta.Key] = declareds
}
declareds[declared] = struct{}{}
}
}
for _, draft := range drafts {
alias := draft.spellings[0]
for _, spelling := range draft.spellings[1:] {
if spelling < alias {
alias = spelling
}
}
if len(draft.byPlugin) != 1 {
pluginKeys := make([]string, 0, len(draft.byPlugin))
for key := range draft.byPlugin {
pluginKeys = append(pluginKeys, key)
}
common.SysLog(common.LogText("task model alias %q dropped: maps to multiple plugins %v", alias, pluginKeys))
continue
}
var pluginKey, declared string
for key, declareds := range draft.byPlugin {
pluginKey = key
if len(declareds) == 1 {
for name := range declareds {
declared = name
}
}
}
view.byFold[jsplugin.ASCIIFold(alias)] = TaskAliasTarget{
Alias: alias,
Declared: declared,
PluginKey: pluginKey,
}
}
return view
}
// followChannelModelMapping walks one channel's mapping the same way
// ModelMappedHelper does: visited-set cycle detection, self-map stops at
// the current hop, a non-self cycle is reported to the caller.
func followChannelModelMapping(modelMap map[string]string, start string) (string, bool) {
current := start
visited := map[string]bool{current: true}
for {
mapped, exists := modelMap[current]
if !exists || mapped == "" {
return current, false
}
if visited[mapped] {
if mapped == current {
return current, false
}
return "", true
}
visited[mapped] = true
current = mapped
}
}