Files
Calcium-Ion 128089fa7d feat(i18n): translate web console messages in the frontend and AI-client errors in the backend (#7712)
* feat(i18n): translate web console messages in the frontend and AI-client errors in the backend

Chinese text was hard-coded in API messages, stored log content, emails and
server logs, so it reached users of every interface language.

Text is now translated where its reader is:

- Web console APIs return the English source text as message_key with
  message_params, and the web console translates it in all seven languages.
  message keeps the English text for other API callers.
- Errors returned to AI clients, emails and notifications are translated by
  the backend (en, zh-CN, zh-TW) in the language saved on the account, then
  the language of the request's Accept-Language header.
- A reader who states no language still gets the original Chinese text of the
  messages that were Chinese before, word for word. Gateways that use another
  new-api as their upstream match that text, for example to disable a channel
  whose account ran out of quota.
- Usage, top-up and system log content is stored as content_parts (key and
  params) and rendered in the viewer's language. Task fail reasons and refund
  reasons written by the gateway are English keys. Rows written earlier are
  shown as before.
- Server logs are English.

Also:

- The saved interface language zhTW, and zh-HK, zh-MO and zh-Hant, now select
  Traditional Chinese in the backend instead of Simplified Chinese.
- The built-in group descriptions are translated; descriptions an
  administrator saved are shown as written.
- .agents/rules/i18n.md records the conventions. Tests in i18n/ fail on a
  non-English string literal, on a web console message without translations
  and on a locale key without a constant.

* feat(i18n): print server logs in Chinese when LOG_LANGUAGE asks for it

Server log lines are English by default. With LOG_LANGUAGE=zh-CN, the lines
that were Chinese before print their original Chinese text.

common.LogText formats a log line from its English format, which is also the
key of its Chinese text in common/log_text.zh-CN.json. A line without an entry
prints in English. go vet checks the arguments of every call, and
TestServerLogTexts checks that each entry still matches a call and has the
same verbs as its key.

* docs: list LOG_LANGUAGE in the docker compose example

* feat(i18n): translate every server log line and add DEFAULT_LANGUAGE

Every line the gateway prints goes through common.LogText. With
DEFAULT_LANGUAGE=zh-CN, the lines that have a Chinese text in
common/log_text.zh-CN.json print it and the others print English.

DEFAULT_LANGUAGE replaces LOG_LANGUAGE. It also sets the language of
backend messages, emails and notifications for a reader who states no
language: en, zh-CN or zh-TW, English when unset. The list of messages
that stayed Chinese for such readers is removed.

Review fixes:
- Channel test logs store the token name "Model test"; the console shows
  it in the interface language.
- Task fail reasons written by the gateway are fixed sentences the console
  translates. The HTTP status and the upstream detail go to the server log
  when a polled task is failed.
- The auto-disable reason of a slow channel is written in DEFAULT_LANGUAGE.

Also:
- Drop the repository-scanning lint tests, the i18n rules file and its
  read gate. AGENTS.md keeps the conventions in four lines.
- Payment error toasts keep their previous text and only add the
  translated message key.
- Remove guards and helpers that no caller needs.

* fix(i18n): address review on language lookup, notices and error responses

- Relay requests no longer load the user a second time to find the
  language: the settings TokenAuth put into the request are used as they
  are, also when they hold no language.
- Upstream model update notices are rendered in each recipient's saved
  language, DEFAULT_LANGUAGE when none.
- /api/usage/token and /api/log/token answer with backend-translated
  messages again, with the keys and status codes used before this PR.
  Their callers are scripts, not the web console.
- Webhook notifications fill {{value}} placeholders like the other
  senders. Running the rendered text through Sprintf appended
  %!(EXTRA ...) and broke %2F in links.
- RootUserLanguage reads the setting of the root user it already loaded.
- zh_TW and zh_HK are read as Traditional Chinese.
- Add common.ApiErrorStatus for error responses under another HTTP status
  and use it where those responses were built by hand.
- Remove middleware.GetLanguage, which had no callers.
2026-10-09 21:39:51 +08:00

281 lines
6.7 KiB
Go

package model
import (
"encoding/json"
"maps"
"slices"
"strings"
"github.com/QuantumNous/new-api/common"
)
const (
logOtherAdminInfoKey = "admin_info"
logOtherRootInfoKey = "root_info"
logOtherAuditInfoKey = "audit_info"
logOtherContentPartsKey = "content_parts"
)
// legacySensitiveLogOtherKeys are historical top-level fields that must never
// be written via SetPublic and must be stripped from user-visible projections.
var legacySensitiveLogOtherKeys = []string{
"channel_id",
"channel_name",
"channel_type",
"reject_reason",
}
type logOtherVisibility int
const (
logOtherVisibilityUser logOtherVisibility = iota
logOtherVisibilityAdmin
logOtherVisibilityRoot
)
// LogOther separates usage-log metadata by the audience allowed to see it.
// Its maps stay private so callers cannot accidentally place privileged fields
// in the user-visible top level.
type LogOther struct {
public map[string]any
adminInfo map[string]any
rootInfo map[string]any
auditInfo map[string]any
}
func NewLogOther() *LogOther {
return &LogOther{}
}
func isReservedLogOtherKey(key string) bool {
switch key {
case logOtherAdminInfoKey, logOtherRootInfoKey, logOtherAuditInfoKey:
return true
default:
return slices.Contains(legacySensitiveLogOtherKeys, key)
}
}
// SetPublic records metadata that log owners may receive from self/token log APIs.
// It rejects role-scoped and legacy-sensitive keys so new writers cannot recreate
// the historical channel/reject-reason leak.
func (o *LogOther) SetPublic(key string, value any) bool {
if o == nil || key == "" || isReservedLogOtherKey(key) {
return false
}
if o.public == nil {
o.public = make(map[string]any)
}
o.public[key] = value
return true
}
func (o *LogOther) MergePublic(values map[string]any) {
for key, value := range values {
o.SetPublic(key, value)
}
}
// setContent stores structured content under content_parts, which the web
// console renders in the viewer's language, and returns the English text kept
// in the content column for exports and API callers.
func (o *LogOther) setContent(parts []*common.Message) string {
if len(parts) == 0 {
return ""
}
o.SetPublic(logOtherContentPartsKey, parts)
texts := make([]string, 0, len(parts))
for _, part := range parts {
texts = append(texts, part.Error())
}
return strings.Join(texts, ", ")
}
func (o *LogOther) SetAdmin(key string, value any) bool {
if o == nil || key == "" {
return false
}
if o.adminInfo == nil {
o.adminInfo = make(map[string]any)
}
o.adminInfo[key] = value
return true
}
func (o *LogOther) MergeAdmin(values map[string]any) {
for key, value := range values {
o.SetAdmin(key, value)
}
}
func (o *LogOther) SetRoot(key string, value any) bool {
if o == nil || key == "" {
return false
}
if o.rootInfo == nil {
o.rootInfo = make(map[string]any)
}
o.rootInfo[key] = value
return true
}
func (o *LogOther) MergeRoot(values map[string]any) {
for key, value := range values {
o.SetRoot(key, value)
}
}
func (o *LogOther) SetAudit(key string, value any) bool {
if o == nil || key == "" {
return false
}
if o.auditInfo == nil {
o.auditInfo = make(map[string]any)
}
o.auditInfo[key] = value
return true
}
func (o *LogOther) MergeAudit(values map[string]any) {
for key, value := range values {
o.SetAudit(key, value)
}
}
func copyLogOtherMap(values map[string]any) map[string]any {
if len(values) == 0 {
return nil
}
copyValues := make(map[string]any, len(values))
maps.Copy(copyValues, values)
return copyValues
}
func (o *LogOther) toMap() map[string]any {
result := make(map[string]any)
if o == nil {
return result
}
maps.Copy(result, o.public)
if adminInfo := copyLogOtherMap(o.adminInfo); len(adminInfo) > 0 {
result[logOtherAdminInfoKey] = adminInfo
}
if auditInfo := copyLogOtherMap(o.auditInfo); len(auditInfo) > 0 {
result[logOtherAuditInfoKey] = auditInfo
}
if rootInfo := copyLogOtherMap(o.rootInfo); len(rootInfo) > 0 {
result[logOtherRootInfoKey] = rootInfo
}
return result
}
func (o *LogOther) jsonString() string {
if o == nil {
return ""
}
data, err := common.Marshal(o.toMap())
if err != nil {
common.SysError(common.LogText("failed to marshal log other: %s", err.Error()))
return ""
}
return string(data)
}
// JSONString returns the complete stored representation, including all
// privileged scopes. API projections must use the role-specific formatter.
func (o *LogOther) JSONString() string {
return o.jsonString()
}
// Snapshot returns a detached top-level view for tests and read-only
// inspection. Mutating it cannot add or replace fields in LogOther.
func (o *LogOther) Snapshot() map[string]any {
if o == nil {
return nil
}
return o.toMap()
}
func (o *LogOther) MarshalJSON() ([]byte, error) {
return common.Marshal(o.toMap())
}
func normalizeLegacyRejectReason(values map[string]json.RawMessage) bool {
rejectReason, ok := values["reject_reason"]
if !ok {
return false
}
adminInfo := make(map[string]json.RawMessage)
if rawAdminInfo, exists := values[logOtherAdminInfoKey]; exists {
_ = common.Unmarshal(rawAdminInfo, &adminInfo)
}
if adminInfo == nil {
adminInfo = make(map[string]json.RawMessage)
}
if _, exists := adminInfo["reject_reason"]; !exists {
adminInfo["reject_reason"] = rejectReason
}
encodedAdminInfo, err := common.Marshal(adminInfo)
if err != nil {
return false
}
values[logOtherAdminInfoKey] = encodedAdminInfo
delete(values, "reject_reason")
return true
}
// formatLogOtherJSON applies the role projection while keeping untouched JSON
// values as RawMessage. This preserves integers larger than JavaScript's safe
// range instead of round-tripping them through float64.
func formatLogOtherJSON(value string, visibility logOtherVisibility) string {
if value == "" {
return ""
}
var values map[string]json.RawMessage
if err := common.UnmarshalJsonStr(value, &values); err != nil {
if visibility == logOtherVisibilityRoot {
return value
}
return "{}"
}
changed := false
if visibility == logOtherVisibilityUser {
for _, key := range []string{logOtherAdminInfoKey, logOtherRootInfoKey, logOtherAuditInfoKey} {
if _, exists := values[key]; exists {
delete(values, key)
changed = true
}
}
for _, key := range legacySensitiveLogOtherKeys {
if _, exists := values[key]; exists {
delete(values, key)
changed = true
}
}
} else {
changed = normalizeLegacyRejectReason(values)
if visibility == logOtherVisibilityAdmin {
if _, exists := values[logOtherRootInfoKey]; exists {
delete(values, logOtherRootInfoKey)
changed = true
}
}
}
if !changed {
return value
}
formatted, err := common.Marshal(values)
if err != nil {
if visibility == logOtherVisibilityRoot {
return value
}
return "{}"
}
return string(formatted)
}