Add feedback service, reminder diagnostics and smooth week transitions

This commit is contained in:
Codex
2026-10-14 15:53:21 +08:00
parent c5e736b39b
commit 1f06e3741e
40 changed files with 3982 additions and 25 deletions
+11
View File
@@ -0,0 +1,11 @@
.env
.env.*
!.env.example
.venv
.deps
.test*
.pytest_cache
__pycache__
data
tests
*.sqlite3*
+17
View File
@@ -0,0 +1,17 @@
# Server only. Never put these secrets in Flutter, the APK, or source control.
SMTP_HOST=smtp.feishu.cn
SMTP_PORT=465
SMTP_TLS=ssl
SMTP_USERNAME=authors@glowtps.cn
SMTP_PASSWORD='REPLACE_WITH_FEISHU_MAIL_CLIENT_AUTHORIZATION_PASSWORD'
MAIL_FROM=authors@glowtps.cn
MAIL_TO=tps-david@glowtps.cn
# Generate once on the server, keep unchanged across container updates/restarts.
IDEMPOTENCY_SECRET='REPLACE_WITH_RANDOM_SECRET_AT_LEAST_32_BYTES'
DATABASE_PATH=/app/data/feedback.sqlite3
SMTP_TIMEOUT_SECONDS=15
RETENTION_DAYS=7
IP_REQUESTS_PER_HOUR=10
GLOBAL_REQUESTS_PER_HOUR=100
MAX_CONCURRENT_REQUESTS=4
UPLOAD_TIMEOUT_SECONDS=60
+10
View File
@@ -0,0 +1,10 @@
.env
.env.*
!.env.example
.venv/
.deps/
.test*/
.pytest_cache/
__pycache__/
data/
*.sqlite3*
+13
View File
@@ -0,0 +1,13 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 PYTHONUNBUFFERED=1
WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt \
&& useradd --system --uid 10001 --create-home feedback \
&& mkdir /app/data && chown feedback:feedback /app/data
COPY feedback_service.py main.py ./
USER feedback
EXPOSE 8000
# The host uses localhost port publishing; only that trusted proxy may supply client IP headers.
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1", "--proxy-headers", "--forwarded-allow-ips", "*", "--no-access-log", "--limit-concurrency", "32", "--timeout-keep-alive", "5", "--timeout-graceful-shutdown", "180"]
+96
View File
@@ -0,0 +1,96 @@
# App feedback → Feishu SMTP
Prepared for Ubuntu + 1Panel on `43.138.57.216`. Public endpoint: `https://app.glowtps.cn/v1/feedback`. Server sender: `authors@glowtps.cn`; fixed receiver: `tps-david@glowtps.cn`. This directory contains deployment source; no connection to the server or live SMTP test has been performed.
## Contract
`POST /v1/feedback` uses `multipart/form-data` only. Each text field occurs once; repeated attachments use the field name `attachments`.
| Field | Problem feedback | Development request |
|---|---|---|
| `submission_id` | Required random identifier, 16–128 characters `[A-Za-z0-9_-]` (UUID v4 works). Keep unchanged for retries. | Same |
| `type` | `problem` | `development` |
| `message` | Required, nonblank, max 5000 Unicode characters | Must be omitted, even if empty |
| `contact` | Optional, max 200 characters | Required, nonblank, max 200 characters |
| `school` | Must be omitted | Required, nonblank, max 200 characters |
| `attachments` | Optional, max 3 files, 8 MiB each, 20 MiB aggregate | Must be omitted |
Accepted MIME types: `image/jpeg`, `image/png`, `image/gif`, `image/webp`, `image/heic`, `image/heif`, `video/mp4`, `video/quicktime`, `video/x-m4v`, `video/webm`. Send an explicit matching MIME type; `application/octet-stream` is rejected. Container signatures must match. MOV/MP4/M4V/HEIC require an initial valid `ftyp` box with a recognized compatible brand; legacy MOV without `ftyp` is rejected. Checks identify file formats; they do not certify an attachment is harmless or a complete playable file.
Filenames lose directory components, control characters and unsupported characters, and are bounded. Nothing is served as public media. All text goes into a plain-text mail body; sender/recipient and mail headers never come from submitted text.
Success is **HTTP 200** with exactly:
```json
{"success":true,"submission_id":"client-generated-id","status":"sent"}
```
This means the configured SMTP server acknowledged message acceptance with `250`, and that status was committed locally. It does not guarantee inbox delivery or reading. Same-ID/same-content retries return the recorded success without sending again. Fingerprints include exact field values, sanitized filenames, MIME types and attachment content in the original order. Retry with the same payload and ID; generate a new ID when editing a previously submitted payload.
All failures use non-2xx responses, with `success:false`, `error`, `detail` and (where applicable) `submission_id`/`status`:
| HTTP | Error/status | Action |
|---|---|---|
| 400 / 408 | Malformed body / upload timeout | Fix request or retry the same ID |
| 413 / 415 / 422 | Upload limits / unsupported media / invalid fields | Fix payload; no mail was attempted |
| 409 | `id_conflict` | This ID already belongs to other content; do not blindly resend |
| 409 | `in_progress` | Wait, then retry the identical ID and payload |
| 429 | `rate_limited` | Respect `Retry-After` |
| 503 | `server_busy` | Respect `Retry-After`; retain the ID |
| 503 | `smtp_unavailable`, status `retryable` | SMTP definitely did not accept mail; retry the same ID |
| 503 | `delivery_uncertain` or `storage_unavailable`, status `uncertain` | Stop automatic resubmission and ask the operator to check the mail by submission ID |
Proxy-generated errors can be HTML rather than this JSON; clients must still treat them as failures. No `202` or queue acceptance counts as success.
## Ubuntu / 1Panel deployment
1. Point the DNS A record for `app.glowtps.cn` to `43.138.57.216`. In Tencent Cloud's security group allow web ports 80/443; keep 18080 closed to the public. Use the existing 1Panel/OpenResty installation.
2. Upload **this directory's deployable files**, without `.deps`, test caches, local database files or any developer environment file, to a dedicated server directory such as `/opt/app-feedback`. In 1Panel's file manager or server terminal, copy `.env.example` to `.env`; set its permissions to `600`.
3. In Feishu Mail, enable third-party mail-client access for `authors@glowtps.cn` and generate the client authorization password. Put it only in `SMTP_PASSWORD` on the server. Use `SMTP_USERNAME=authors@glowtps.cn`, `MAIL_FROM=authors@glowtps.cn` and `MAIL_TO=tps-david@glowtps.cn`. Single-quote secret values in `.env` so `$` is literal in Compose. Do not paste passwords into chat, Flutter code, Dockerfiles, terminal command history or screenshots.
4. Generate `IDEMPOTENCY_SECRET` once on the server (for example `openssl rand -hex 32`), insert it into `.env` and keep it stable across updates. Changing this key while retaining the database makes existing fingerprints conflict. Replace both placeholder secrets before starting.
5. From the uploaded directory run `docker compose up -d --build` (or create a 1Panel Compose project from `docker-compose.yml` with this directory as its build context). The image runs as an unprivileged user, the filesystem is read-only except the named SQLite volume and temporary upload space, and the host publishes only `127.0.0.1:18080`.
6. Verify liveness locally with `curl --fail http://127.0.0.1:18080/healthz` and check `docker compose ps`. Health checks do **not** send mail or validate SMTP credentials. If startup fails, inspect container logs; the app validates required configuration without printing values. Do not share `docker compose config` or `docker inspect` output, because they can expose credentials.
7. In 1Panel, create/configure the `app.glowtps.cn` reverse-proxy website, obtain/select its HTTPS certificate and enable HTTPS redirection. Configure the upstream as `http://127.0.0.1:18080`. Add the location settings from `reverse-proxy.conf.example` inside the site's generated HTTPS server block, preserving its certificate paths. If the site already exists, add only the feedback and health locations rather than replacing other routes. In the generated config, avoid duplicate `location = /v1/feedback` blocks. Test OpenResty's configuration through 1Panel before reload.
8. Check that 1Panel's OpenResty has **host network access**. A proxy container on a bridge network has its own `127.0.0.1`; it cannot reach a localhost-only host publish. With that setup, configure a shared private Docker network and proxy to `feedback:8000`, remove the host port publish, and restrict proxy trust to that network instead of exposing 18080 publicly. The supplied example targets the host-access configuration; inspect the existing network mode before use.
9. Verify `https://app.glowtps.cn/healthz`. When explicitly ready to send a real message, submit one controlled feedback, confirm the response and receipt at `tps-david@glowtps.cn`, then retry the **same ID and payload** to verify only one mail arrives. That real-mail check remains to be done by the operator.
Feishu's official documentation specifies `smtp.feishu.cn` with SSL on 465, and its business-system guide documents STARTTLS on 587. The service supports only verified TLS: `SMTP_PORT=465` + `SMTP_TLS=ssl` (default), or `SMTP_PORT=587` + `SMTP_TLS=starttls`. It never falls back to plaintext or disables certificate checks. Outbound 465/587 access must be permitted by the server/network.
## Durable retries and retention
SQLite in the named `feedback-data` volume stores only submission ID, a keyed content fingerprint, state and timestamps. It stores no form text, contact address, school, filename or raw attachment. Uploaded data is spooled to bounded temporary files and deleted on request completion; in Docker, the temporary area is a tmpfs. The resulting email and attachments are retained by Feishu/recipient mailbox policies, independently of this database.
There is no atomic transaction across SMTP and SQLite. A lost `DATA` acknowledgement or crash after acceptance may mean mail arrived even though the HTTP request failed. Such records are quarantined as `uncertain`, preventing automatic resend. A `sending` record stays `in_progress` for five minutes; afterwards it becomes `uncertain`. The stable email Message-ID includes the submission ID, but recipient deduplication is not assumed.
Metadata expires after seven days since the last state change (`RETENTION_DAYS`). Cleanup runs at startup, on submissions, and hourly while the process runs. The retry deduplication guarantee is bounded by this retention window; reusing an ID after expiry can send again. Client automatic retries must stop within the retention window. Keep the volume and key across deployments, and apply an appropriate retention policy to any backups. Use `docker compose down` for a stop; **do not use `down -v`** when preserving retry records. Back up SQLite consistently while the service is stopped or using SQLite's backup API, including its WAL requirements.
To resolve an uncertain record, first check the recipient mailbox and provider delivery records for its submission ID. During a maintenance stop, use a trusted SQLite tool to inspect `submissions`. Change only that record's `status` to `sent` after verified acceptance, or to `retryable` only after verified non-acceptance, updating `updated` to the current Unix timestamp. Never automatically clear uncertain records, delete the volume or issue a new submission ID to force a retry.
## Limits and operations
The service counts raw HTTP bytes, including chunked input, with a 20 MiB + 64 KiB request ceiling; the proxy allows 21 MiB to leave the final precise check to the service. Text parts, field count and file count are bounded. Four active feedback requests are allowed by default, with a 60-second upload deadline. Default limits are ten requests/IP/hour and 100 requests globally/hour. These include invalid requests and retries. The in-memory rate counters reset after restart; SQLite deduplication survives. Proxy IP headers are overwritten; the private upstream must stay private.
Run exactly **one worker and one service replica** with this configuration: rate/concurrency counters are per-process. SQLite claims serialize concurrent IDs, but scaling needs a shared rate limiter and an appropriate storage design. The default 512 MiB memory and 256 MiB tmpfs budget account for multipart and MIME encoding of several uploads; if reducing these, also reduce concurrency. Large attachments can grow by roughly a third in email MIME encoding and may be rejected by provider message-size limits; a definite rejection returns a failure.
The app logs neither user fields nor SMTP responses/secrets; Uvicorn access logs are disabled. Feedback-path proxy access logs are also disabled in the example. Do not enable SMTP debug logging, request-body/WAF capture or form-content tracing on this endpoint. Allowlisted media are attached to plain-text mail only and never executed by the service. For higher abuse exposure, add server-side account verification/CAPTCHA through the app's existing identity system; no static secret embedded in an APK provides meaningful access control.
## Local verification (no real SMTP)
Requires Python 3.11+; the supplied image uses 3.12. Install dependencies into a virtual environment in this directory:
```sh
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements-dev.txt
python -m pytest -q
```
Tests inject a recording sender or scripted SMTP protocol object. They cover type-specific validation, file MIME/signatures, size/count and chunked-body limits, definite/uncertain SMTP outcomes, verified SSL/STARTTLS order, recipient fixation, persistence/restarts, conflicting IDs, concurrent retries, storage-commit failure, rate/concurrency limits, and metadata privacy. No SMTP connection, credentials or live mail is needed.
## References
- [Feishu: third-party mail client login](https://www.feishu.cn/hc/en-US/articles/902478147400-log-in-to-feishu-mail-through-a-third-party-email-client)
- [Feishu: SMTP migration / 465 SSL and 587 STARTTLS](https://www.feishu.cn/hc/en-US/articles/221905160046-update-the-smtp-server-settings-in-business-systems-during-the-email-migration-process)
- [1Panel: create websites and reverse proxies](https://docs.fit2cloud.com/1panel/user_manual/websites/website-create/)
- [Starlette: bounded multipart form parsing](https://starlette.dev/requests/)
- [Docker Compose: quoted environment values](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/)
+34
View File
@@ -0,0 +1,34 @@
name: app-feedback
services:
feedback:
build: .
restart: unless-stopped
env_file:
- .env
ports:
- "127.0.0.1:18080:8000"
volumes:
- feedback-data:/app/data
read_only: true
tmpfs:
- /tmp:size=256m,mode=1777
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
mem_limit: 512m
cpus: 1.0
pids_limit: 64
stop_grace_period: 180s
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/healthz', timeout=3).read()"]
interval: 30s
timeout: 5s
retries: 3
logging:
driver: json-file
options:
max-size: "5m"
max-file: "2"
volumes:
feedback-data:
+513
View File
@@ -0,0 +1,513 @@
"""Feedback submission service. Never log user text, uploads, or SMTP secrets."""
import asyncio
import hashlib
import hmac
import json
import os
import re
import smtplib
import sqlite3
import ssl
import tempfile
import threading
import time
from collections import deque
from contextlib import asynccontextmanager, contextmanager, suppress
from dataclasses import dataclass, field
from email.message import EmailMessage
from email.policy import SMTP
from email.utils import formatdate
from pathlib import Path
from fastapi import FastAPI, Request
from starlette.datastructures import UploadFile
from starlette.exceptions import HTTPException
from starlette.responses import JSONResponse
from starlette.concurrency import run_in_threadpool
MIB = 1024 * 1024
FILE_LIMIT = 8 * MIB
TOTAL_FILE_LIMIT = 20 * MIB
BODY_LIMIT = TOTAL_FILE_LIMIT + 64 * 1024
ID_PATTERN = re.compile(r"[A-Za-z0-9_-]{16,128}\Z")
ADDRESS_PATTERN = re.compile(r"[A-Za-z0-9._%+-]+@(?:[A-Za-z0-9-]+\.)+[A-Za-z]{2,63}\Z")
@dataclass
class Settings:
database_path: str = "data/feedback.sqlite3"
smtp_host: str = "smtp.feishu.cn"
smtp_port: int = 465
smtp_tls: str = "ssl"
smtp_username: str = ""
smtp_password: str = field(default="", repr=False)
mail_from: str = ""
mail_to: str = ""
idempotency_secret: str = field(default="", repr=False)
smtp_timeout_seconds: int = 15
retention_days: int = 7
inflight_stale_seconds: int = 300
ip_requests_per_hour: int = 10
global_requests_per_hour: int = 100
max_concurrent_requests: int = 4
upload_timeout_seconds: int = 60
@classmethod
def from_env(cls):
names = {
"database_path": "DATABASE_PATH", "smtp_host": "SMTP_HOST", "smtp_port": "SMTP_PORT",
"smtp_tls": "SMTP_TLS", "smtp_username": "SMTP_USERNAME", "smtp_password": "SMTP_PASSWORD",
"mail_from": "MAIL_FROM", "mail_to": "MAIL_TO", "idempotency_secret": "IDEMPOTENCY_SECRET",
"smtp_timeout_seconds": "SMTP_TIMEOUT_SECONDS", "retention_days": "RETENTION_DAYS",
"ip_requests_per_hour": "IP_REQUESTS_PER_HOUR", "global_requests_per_hour": "GLOBAL_REQUESTS_PER_HOUR",
"max_concurrent_requests": "MAX_CONCURRENT_REQUESTS", "upload_timeout_seconds": "UPLOAD_TIMEOUT_SECONDS",
}
defaults, values = cls(), {}
for name, env_name in names.items():
if env_name in os.environ:
value = os.environ[env_name]
if isinstance(getattr(defaults, name), int):
try:
value = int(value)
except ValueError:
raise ValueError(f"{env_name} must be an integer") from None
values[name] = value
return cls(**values)
def validate(self):
for name in ("mail_from", "mail_to"):
if not ADDRESS_PATTERN.fullmatch(getattr(self, name)):
raise ValueError(f"{name} must be one plain email address")
if not self.smtp_username.strip() or "\r" in self.smtp_username or "\n" in self.smtp_username:
raise ValueError("SMTP_USERNAME is required and must not contain newlines")
if not self.smtp_password or self.smtp_password.startswith("REPLACE_"):
raise ValueError("SMTP_PASSWORD is required")
if not re.fullmatch(r"[A-Za-z0-9.-]{1,253}", self.smtp_host):
raise ValueError("SMTP_HOST must be a hostname")
if (self.smtp_port, self.smtp_tls) not in {(465, "ssl"), (587, "starttls")}:
raise ValueError("Use SMTP_PORT=465/SMTP_TLS=ssl or SMTP_PORT=587/SMTP_TLS=starttls")
if len(self.idempotency_secret.encode()) < 32 or self.idempotency_secret.startswith("REPLACE_"):
raise ValueError("IDEMPOTENCY_SECRET must have at least 32 bytes")
for name in ("retention_days", "smtp_timeout_seconds", "inflight_stale_seconds",
"ip_requests_per_hour", "global_requests_per_hour", "max_concurrent_requests",
"upload_timeout_seconds"):
if getattr(self, name) <= 0:
raise ValueError(f"{name} must be positive")
if self.smtp_timeout_seconds > 30 or self.max_concurrent_requests > 16:
raise ValueError("SMTP timeout must be <=30 seconds and concurrency <=16")
class ApiError(Exception):
def __init__(self, http_status, error, detail, *, submission_id=None, status=None):
self.http_status, self.error, self.detail = http_status, error, detail
self.submission_id, self.status = submission_id, status
class SendFailure(Exception):
"""SMTP definitely did not accept this message. Retrying is safe."""
class UncertainFailure(Exception):
"""DATA may have been accepted, but its acknowledgement was lost."""
def error_response(code, error, detail, *, submission_id=None, status=None, headers=None):
data = {"success": False, "error": error, "detail": detail}
if submission_id is not None:
data["submission_id"] = submission_id
if status is not None:
data["status"] = status
return JSONResponse(data, status_code=code, headers=headers)
class SmtpSender:
def __init__(self, settings):
self.settings = settings
def __call__(self, message):
config, client, in_data, accepted = self.settings, None, False, False
raw_message = message.as_bytes(policy=SMTP)
try:
context = ssl.create_default_context()
if config.smtp_tls == "ssl":
client = smtplib.SMTP_SSL(config.smtp_host, config.smtp_port,
timeout=config.smtp_timeout_seconds, context=context)
else:
client = smtplib.SMTP(config.smtp_host, config.smtp_port, timeout=config.smtp_timeout_seconds)
client.ehlo()
client.starttls(context=context)
client.ehlo()
client.login(config.smtp_username, config.smtp_password)
code, _ = client.mail(config.mail_from)
if code != 250:
raise SendFailure()
code, _ = client.rcpt(config.mail_to)
if code not in (250, 251):
raise SendFailure()
# A lost response from DATA is ambiguous: never automatically resend.
in_data = True
code, _ = client.data(raw_message)
if code != 250:
raise SendFailure()
accepted = True
except SendFailure:
raise
except smtplib.SMTPDataError:
raise SendFailure() from None
except (OSError, smtplib.SMTPException):
if in_data:
raise UncertainFailure() from None
raise SendFailure() from None
finally:
if client is not None:
try:
if accepted:
client.quit()
else:
client.close()
except (OSError, smtplib.SMTPException):
# QUIT failures do not undo an acknowledged DATA acceptance.
client.close()
class SubmissionStore:
def __init__(self, settings):
self.settings = settings
Path(settings.database_path).parent.mkdir(parents=True, exist_ok=True)
with self.connect() as db:
db.execute("PRAGMA journal_mode=WAL")
db.execute("CREATE TABLE IF NOT EXISTS submissions ("
"submission_id TEXT PRIMARY KEY, digest TEXT NOT NULL, status TEXT NOT NULL, "
"created REAL NOT NULL, updated REAL NOT NULL)")
db.execute("CREATE INDEX IF NOT EXISTS submissions_updated ON submissions(updated)")
self.prune()
@contextmanager
def connect(self):
db = sqlite3.connect(self.settings.database_path, timeout=5)
try:
db.execute("PRAGMA synchronous=FULL")
db.execute("PRAGMA secure_delete=ON")
with db:
yield db
finally:
db.close()
def claim(self, submission_id, digest):
now = time.time()
with self.connect() as db:
db.execute("BEGIN IMMEDIATE")
db.execute("DELETE FROM submissions WHERE updated < ?",
(now - self.settings.retention_days * 86400,))
row = db.execute("SELECT digest, status, updated FROM submissions WHERE submission_id=?",
(submission_id,)).fetchone()
if row:
old_digest, status, updated = row
if not hmac.compare_digest(old_digest, digest):
return "conflict"
if status == "sending":
if now - updated <= self.settings.inflight_stale_seconds:
return "in_progress"
db.execute("UPDATE submissions SET status='uncertain', updated=? WHERE submission_id=?",
(now, submission_id))
return "uncertain"
if status in ("sent", "uncertain"):
return status
db.execute("UPDATE submissions SET status='sending', updated=? WHERE submission_id=?",
(now, submission_id))
else:
db.execute("INSERT INTO submissions VALUES (?, ?, 'sending', ?, ?)",
(submission_id, digest, now, now))
return "claimed"
def finish(self, submission_id, status):
with self.connect() as db:
db.execute("UPDATE submissions SET status=?, updated=? WHERE submission_id=?",
(status, time.time(), submission_id))
def prune(self):
with self.connect() as db:
db.execute("DELETE FROM submissions WHERE updated < ?",
(time.time() - self.settings.retention_days * 86400,))
class AdmissionControl:
"""Bounded per-process admission state: run exactly one worker."""
def __init__(self, settings):
self.settings, self.lock, self.current = settings, threading.Lock(), 0
self.global_times, self.ip_times = deque(), {}
def enter(self, ip):
now = time.monotonic()
ip_key = hmac.new(self.settings.idempotency_secret.encode(), ip.encode(), hashlib.sha256).digest()
with self.lock:
while self.global_times and self.global_times[0] <= now - 3600:
self.global_times.popleft()
self.ip_times = {k: times for k, times in self.ip_times.items() if times[-1] > now - 3600}
times = self.ip_times.get(ip_key, deque())
while times and times[0] <= now - 3600:
times.popleft()
if (len(times) >= self.settings.ip_requests_per_hour or
len(self.global_times) >= self.settings.global_requests_per_hour):
return error_response(429, "rate_limited", "Submission limit reached; try again later.",
headers={"Retry-After": "3600"})
if self.current >= self.settings.max_concurrent_requests:
return error_response(503, "server_busy", "Server is busy; retry with the same submission ID.",
headers={"Retry-After": "15"})
self.ip_times[ip_key] = times
times.append(now)
self.global_times.append(now)
self.current += 1
return None
def leave(self):
with self.lock:
self.current -= 1
class BoundedFeedbackBody:
"""Count raw bytes before parsing, including chunked bodies and multipart overhead."""
def __init__(self, app, settings, admission):
self.app, self.settings, self.admission = app, settings, admission
async def __call__(self, scope, receive, send):
if scope["type"] != "http" or scope.get("path") != "/v1/feedback" or scope["method"] != "POST":
return await self.app(scope, receive, send)
denial = self.admission.enter((scope.get("client") or ("unknown",))[0])
if denial:
return await denial(scope, receive, send)
try:
headers = dict(scope.get("headers", []))
if headers.get(b"content-type", b"").split(b";", 1)[0].strip().lower() != b"multipart/form-data":
return await error_response(415, "unsupported_content_type", "Use multipart/form-data.")(scope, receive, send)
if b"content-length" in headers:
try:
length = int(headers[b"content-length"])
except ValueError:
return await error_response(400, "invalid_body", "Invalid Content-Length.")(scope, receive, send)
if length < 0:
return await error_response(400, "invalid_body", "Invalid Content-Length.")(scope, receive, send)
if length > BODY_LIMIT:
return await error_response(413, "upload_too_large", "Request exceeds upload limits.")(scope, receive, send)
with tempfile.SpooledTemporaryFile(max_size=MIB, mode="w+b") as spool:
total, deadline = 0, time.monotonic() + self.settings.upload_timeout_seconds
while True:
remaining = deadline - time.monotonic()
if remaining <= 0:
return await error_response(408, "upload_timeout", "Upload timed out.")(scope, receive, send)
try:
event = await asyncio.wait_for(receive(), remaining)
except TimeoutError:
return await error_response(408, "upload_timeout", "Upload timed out.")(scope, receive, send)
if event["type"] == "http.disconnect":
return
body = event.get("body", b"")
total += len(body)
if total > BODY_LIMIT:
return await error_response(413, "upload_too_large", "Request exceeds upload limits.")(scope, receive, send)
spool.write(body)
if not event.get("more_body", False):
break
spool.seek(0)
replayed = False
async def replay_receive():
nonlocal replayed
if replayed:
return {"type": "http.disconnect"}
block = spool.read(64 * 1024)
replayed = spool.tell() >= total
return {"type": "http.request", "body": block, "more_body": not replayed}
return await self.app(scope, replay_receive, send)
finally:
self.admission.leave()
def file_signature(data, mime):
if mime == "image/jpeg":
return data.startswith(b"\xff\xd8\xff"), ".jpg"
if mime == "image/png":
return data.startswith(b"\x89PNG\r\n\x1a\n"), ".png"
if mime == "image/gif":
return data.startswith((b"GIF87a", b"GIF89a")), ".gif"
if mime == "image/webp":
return len(data) >= 12 and data[:4] == b"RIFF" and data[8:12] == b"WEBP", ".webp"
if mime == "video/webm":
return data.startswith(b"\x1a\x45\xdf\xa3") and b"webm" in data[:4096], ".webm"
if mime in {"image/heic", "image/heif", "video/mp4", "video/quicktime", "video/x-m4v"}:
if len(data) < 16 or data[4:8] != b"ftyp":
return False, ""
box_size = int.from_bytes(data[:4], "big")
if box_size < 16 or box_size > len(data) or box_size > 4096 or box_size % 4:
return False, ""
brands = {data[8:12]} | {data[i:i + 4] for i in range(16, box_size, 4)}
if mime in {"image/heic", "image/heif"}:
return bool(brands & {b"heic", b"heix", b"hevc", b"hevx", b"heim", b"heis", b"hevm", b"hevs"}), ".heic"
if mime == "video/quicktime":
return b"qt " in brands, ".mov"
return bool(brands & {b"isom", b"iso2", b"avc1", b"mp41", b"mp42", b"M4V ", b"M4VH", b"M4VP", b"dash", b"MSNV"}), ".mp4"
return False, ""
def safe_filename(filename, extension):
filename = (filename or "attachment").replace("\\", "/").rsplit("/", 1)[-1]
filename = re.sub(r"[^A-Za-z0-9._-]", "_", filename).strip("._")[:100] or "attachment"
alternatives = {".jpg": {".jpg", ".jpeg"}, ".heic": {".heic", ".heif"}, ".mp4": {".mp4", ".m4v"}}
if Path(filename).suffix.lower() not in alternatives.get(extension, {extension}):
filename = Path(filename).stem + extension
return filename
async def parse_submission(request):
try:
async with request.form(max_files=3, max_fields=5, max_part_size=20000) as form:
fields, uploads = {}, []
for name, value in form.multi_items():
if isinstance(value, UploadFile):
if name != "attachments":
raise ApiError(422, "validation_error", "Unexpected file field.")
uploads.append(value)
else:
if name not in {"submission_id", "type", "message", "contact", "school"} or name in fields:
raise ApiError(422, "validation_error", "Unexpected or duplicated form field.")
fields[name] = value
submission_id = fields.get("submission_id", "")
if not ID_PATTERN.fullmatch(submission_id):
raise ApiError(422, "validation_error", "submission_id must be a random 16–128 character identifier.")
kind = fields.get("type")
if kind not in {"problem", "development"}:
raise ApiError(422, "validation_error", "type must be problem or development.")
for name, maximum in (("message", 5000), ("contact", 200), ("school", 200)):
if len(fields.get(name, "")) > maximum:
raise ApiError(422, "validation_error", f"{name} exceeds {maximum} characters.")
if any("\x00" in text for text in fields.values()):
raise ApiError(422, "validation_error", "Null characters are not allowed.")
if kind == "problem":
if not fields.get("message", "").strip() or "school" in fields:
raise ApiError(422, "validation_error", "Problem feedback requires message and must omit school.")
elif ("message" in fields or uploads or not fields.get("school", "").strip()
or not fields.get("contact", "").strip()):
raise ApiError(422, "validation_error", "Development requests require school and contact; omit message and attachments.")
attachments, total = [], 0
for upload in uploads:
data = await upload.read(FILE_LIMIT + 1)
total += len(data)
if len(data) > FILE_LIMIT or total > TOTAL_FILE_LIMIT:
raise ApiError(413, "upload_too_large", "Maximum three files, 8 MiB each and 20 MiB total.")
mime = (upload.content_type or "").lower()
valid, extension = file_signature(data, mime)
if not valid:
raise ApiError(415, "unsupported_media", "Attachment MIME type and file signature must match a supported image or video.")
attachments.append((safe_filename(upload.filename, extension), mime, data))
return fields, attachments
except HTTPException as exc:
detail = str(exc.detail).lower()
if "too many" in detail or "maximum" in detail or "exceeded maximum size" in detail:
raise ApiError(413, "upload_too_large", "Multipart field or upload limits exceeded.") from None
raise ApiError(400, "invalid_body", "Malformed multipart body.") from None
def make_message(settings, fields, attachments):
message = EmailMessage(policy=SMTP)
message["From"], message["To"] = settings.mail_from, settings.mail_to
message["Subject"] = f"[App feedback] {fields['type']} / {fields['submission_id']}"
message["Message-ID"] = f"<{fields['submission_id']}@{settings.mail_from.split('@')[1]}>"
message["Date"] = formatdate(localtime=False, usegmt=True)
lines = [f"Submission ID: {fields['submission_id']}", f"Type: {fields['type']}"]
if fields["type"] == "development":
lines += ["School:", fields["school"], "Contact:", fields["contact"]]
else:
lines += ["Contact:", fields.get("contact", ""), "Message:", fields["message"]]
message.set_content("\n".join(lines), charset="utf-8")
for name, mime, data in attachments:
main, sub = mime.split("/", 1)
message.add_attachment(data, maintype=main, subtype=sub, filename=name)
return message
def create_app(settings=None, sender=None):
settings = settings or Settings.from_env()
settings.validate()
store = SubmissionStore(settings)
@asynccontextmanager
async def lifespan(app):
async def cleanup():
while True:
await asyncio.sleep(3600)
# Persistent storage failure makes readiness fail instead of exposing data in logs.
try:
await run_in_threadpool(store.prune)
app.state.retention_healthy = True
except sqlite3.Error:
app.state.retention_healthy = False
app.state.retention_healthy = True
task = asyncio.create_task(cleanup())
try:
yield
finally:
task.cancel()
with suppress(asyncio.CancelledError):
await task
app = FastAPI(docs_url=None, redoc_url=None, openapi_url=None, lifespan=lifespan)
app.state.sender = sender or SmtpSender(settings)
app.add_middleware(BoundedFeedbackBody, settings=settings, admission=AdmissionControl(settings))
@app.exception_handler(ApiError)
async def api_error_handler(request, exc):
return error_response(exc.http_status, exc.error, exc.detail,
submission_id=exc.submission_id, status=exc.status)
@app.get("/healthz")
async def health():
# Liveness only: never sends email or claims SMTP readiness.
if not getattr(app.state, "retention_healthy", True):
return error_response(503, "storage_unavailable", "Retention cleanup requires operator attention.")
return {"status": "ok"}
@app.post("/v1/feedback")
async def feedback(request: Request):
fields, attachments = await parse_submission(request)
submission_id = fields["submission_id"]
canonical = json.dumps({"fields": fields, "attachments": [
{"name": name, "mime": mime, "sha256": hashlib.sha256(data).hexdigest()}
for name, mime, data in attachments]}, sort_keys=True, ensure_ascii=False, separators=(",", ":"))
digest = hmac.new(settings.idempotency_secret.encode(), canonical.encode(), hashlib.sha256).hexdigest()
message = make_message(settings, fields, attachments)
def deliver():
try:
outcome = store.claim(submission_id, digest)
if outcome == "conflict":
raise ApiError(409, "id_conflict", "This ID was already used with different content.", submission_id=submission_id)
if outcome == "in_progress":
raise ApiError(409, "in_progress", "A submission with this ID is still being processed.",
submission_id=submission_id, status="in_progress")
if outcome == "uncertain":
raise ApiError(503, "delivery_uncertain", "Mail acceptance is uncertain; do not submit a new ID. Contact the operator.",
submission_id=submission_id, status="uncertain")
if outcome == "sent":
return
try:
app.state.sender(message)
except SendFailure:
store.finish(submission_id, "retryable")
raise ApiError(503, "smtp_unavailable", "Mail was not accepted; retry with the same ID.",
submission_id=submission_id, status="retryable") from None
except Exception:
store.finish(submission_id, "uncertain")
raise ApiError(503, "delivery_uncertain", "Mail acceptance is uncertain; do not submit a new ID. Contact the operator.",
submission_id=submission_id, status="uncertain") from None
store.finish(submission_id, "sent")
except sqlite3.Error:
raise ApiError(503, "storage_unavailable", "Submission state could not be confirmed; retry only with the same ID.",
submission_id=submission_id, status="uncertain") from None
await run_in_threadpool(deliver)
return {"success": True, "submission_id": submission_id, "status": "sent"}
return app
+3
View File
@@ -0,0 +1,3 @@
from feedback_service import create_app
app = create_app()
+3
View File
@@ -0,0 +1,3 @@
-r requirements.txt
pytest==9.1.1
httpx2==2.13.1
+15
View File
@@ -0,0 +1,15 @@
fastapi==0.143.0
starlette==1.8.0
uvicorn==0.54.0
python-multipart==0.0.32
anyio==4.15.1
pydantic==2.14.1
pydantic-core==2.50.1
typing-extensions==4.16.0
typing-inspection==0.4.4
annotated-doc==0.0.5
annotated-types==0.8.0
opentelemetry-api==1.45.1
click==8.5.0
h11==0.16.0
idna==3.20
@@ -0,0 +1,32 @@
# Insert these locations inside the HTTPS server block for app.glowtps.cn.
# Keep 1Panel's generated certificate paths/listen/server_name directives.
# This assumes OpenResty can access the host's 127.0.0.1 (host network mode).
# Set the site to force HTTPS. Do not expose the upstream port publicly.
location = /v1/feedback {
client_max_body_size 21m;
client_body_timeout 60s;
# Avoid persistent proxy copies of feedback media.
proxy_request_buffering off;
proxy_buffering off;
proxy_http_version 1.1;
proxy_pass http://127.0.0.1:18080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
# Overwrite any header sent by the client so IP rate limiting cannot be spoofed.
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Connection "";
proxy_connect_timeout 5s;
proxy_send_timeout 180s;
proxy_read_timeout 180s;
# NEVER retry a feedback POST against another upstream automatically.
proxy_next_upstream off;
# Disable access logs for this path (no query strings, client IPs or form content).
access_log off;
}
location = /healthz {
proxy_pass http://127.0.0.1:18080;
access_log off;
}
+500
View File
@@ -0,0 +1,500 @@
import asyncio
import sqlite3
import threading
import tempfile
import time
import smtplib
from types import SimpleNamespace
import feedback_service
import pytest
from fastapi.testclient import TestClient
from feedback_service import Settings, SendFailure, UncertainFailure, SmtpSender, SubmissionStore, create_app
ID = "cbaaaace-60b2-4871-8a95-ae316444a2fb"
PNG = b"\x89PNG\r\n\x1a\n" + b"safe fixture"
class RecordingSender:
def __init__(self):
self.messages = []
self.failure = None
def __call__(self, message):
if self.failure:
raise self.failure
self.messages.append(message)
@pytest.fixture
def service(tmp_path, monkeypatch):
monkeypatch.setattr(tempfile, "tempdir", str(tmp_path))
settings = Settings(database_path=str(tmp_path / "feedback.sqlite3"),
smtp_username="feedback@example.com", smtp_password="test-only",
mail_from="feedback@example.com", mail_to="owner@example.com",
idempotency_secret="test-key-at-least-thirty-two-characters")
sender = RecordingSender()
app = create_app(settings, sender)
with TestClient(app) as client:
yield client, sender, settings
def post(client, **overrides):
fields = {"submission_id": ID, "type": "problem", "message": "The timetable is missing."}
fields.update(overrides)
return client.post("/v1/feedback", files=[(k, (None, v)) for k, v in fields.items()])
def test_success_only_after_sender_accepted_and_retry_is_durable(service):
client, sender, settings = service
response = post(client, contact="student@example.com")
assert response.status_code == 200
assert response.json() == {"success": True, "submission_id": ID, "status": "sent"}
assert len(sender.messages) == 1
assert sender.messages[0]["From"] == "feedback@example.com"
assert sender.messages[0]["To"] == "owner@example.com"
assert "student@example.com" in sender.messages[0].get_body().get_content()
# A new application instance reads the persisted status, not an in-memory cache.
with TestClient(create_app(settings, sender)) as restarted:
assert post(restarted, contact="student@example.com").json()["status"] == "sent"
assert len(sender.messages) == 1
def test_reusing_id_with_other_content_rejected(service):
client, sender, _ = service
assert post(client).status_code == 200
assert post(client, message="Changed complaint").status_code == 409
assert len(sender.messages) == 1
@pytest.mark.parametrize("fields", [
{"message": ""}, {"message": "x" * 5001}, {"contact": "x" * 201},
{"school": "unexpected"}, {"type": "other"}, {"submission_id": "too-short"},
{"type": "development", "school": "Example school", "contact": ""},
{"type": "development", "school": "", "contact": "12345678"},
{"type": "development", "school": "Example school", "contact": "12345678"},
])
def test_invalid_fields_do_not_send(service, fields):
client, sender, _ = service
assert post(client, **fields).status_code == 422
assert not sender.messages
def test_development_has_only_school_and_contact(service):
client, sender, _ = service
response = client.post("/v1/feedback", files=[
("submission_id", (None, ID)), ("type", (None, "development")),
("school", (None, "Example school")), ("contact", (None, "student@example.com")),
])
assert response.status_code == 200
body = sender.messages[0].get_body().get_content()
assert "Example school" in body and "student@example.com" in body
def test_attachment_is_sanitized_and_type_checked(service):
client, sender, _ = service
files = [("submission_id", (None, ID)), ("type", (None, "problem")),
("message", (None, "Attachment")),
("attachments", ("../../bad\r\nname.png", PNG, "image/png"))]
response = client.post("/v1/feedback", files=files)
assert response.status_code == 200
attachment = list(sender.messages[0].iter_attachments())[0]
assert attachment.get_filename() == "bad_0D_0Aname.png"
assert attachment.get_payload(decode=True) == PNG
files[-1] = ("attachments", ("pretend.png", b"<script>alert(1)</script>", "image/png"))
assert client.post("/v1/feedback", files=files).status_code == 415
def test_development_rejects_media(service):
client, sender, _ = service
response = client.post("/v1/feedback", files=[
("submission_id", (None, ID)), ("type", (None, "development")),
("school", (None, "Example")), ("contact", (None, "student@example.com")),
("attachments", ("a.png", PNG, "image/png")),
])
assert response.status_code == 422
assert not sender.messages
def test_duplicate_fields_and_unexpected_fields_rejected(service):
client, sender, _ = service
for extra in [("message", (None, "second")), ("recipient", (None, "victim@example.com"))]:
response = client.post("/v1/feedback", files=[
("submission_id", (None, ID)), ("type", (None, "problem")),
("message", (None, "Complaint")), extra,
])
assert response.status_code == 422
assert not sender.messages
def test_definite_smtp_failure_is_non_success_and_can_retry(service):
client, sender, _ = service
sender.failure = SendFailure()
response = post(client)
assert response.status_code == 503
assert response.json()["success"] is False
assert response.json()["status"] == "retryable"
sender.failure = None
assert post(client).json()["status"] == "sent"
def test_uncertain_smtp_acceptance_never_auto_resends(service):
client, sender, settings = service
sender.failure = UncertainFailure()
response = post(client)
assert response.status_code == 503
assert response.json()["status"] == "uncertain"
sender.failure = None
with TestClient(create_app(settings, sender)) as restarted:
response = post(restarted)
assert response.status_code == 503
assert response.json()["error"] == "delivery_uncertain"
assert not sender.messages
def test_parallel_retry_does_not_send_twice(service):
client, sender, _ = service
started, finish = threading.Event(), threading.Event()
def blocking_sender(message):
started.set()
assert finish.wait(10)
sender(message)
client.app.state.sender = blocking_sender
result = []
worker = threading.Thread(target=lambda: result.append(post(client)))
worker.start()
assert started.wait(10)
try:
response = post(client)
assert response.status_code == 409
assert response.json()["status"] == "in_progress"
finally:
finish.set()
worker.join(10)
assert result[0].status_code == 200
assert len(sender.messages) == 1
def test_stale_inflight_record_is_uncertain(service):
client, sender, settings = service
assert post(client).status_code == 200
with sqlite3.connect(settings.database_path) as db:
db.execute("UPDATE submissions SET status='sending', updated=?", (time.time() - 600,))
response = post(client)
assert response.status_code == 503
assert response.json()["status"] == "uncertain"
assert len(sender.messages) == 1
def test_file_count_and_individual_size_are_limited(service):
client, sender, _ = service
base = [("submission_id", (None, ID)), ("type", (None, "problem")),
("message", (None, "Complaint"))]
response = client.post("/v1/feedback", files=base + [
("attachments", (f"{i}.png", PNG, "image/png")) for i in range(4)])
assert response.status_code == 413
response = client.post("/v1/feedback", files=base + [
("attachments", ("large.png", PNG + b"x" * (8 * 1024 * 1024), "image/png"))])
assert response.status_code == 413
assert not sender.messages
def test_aggregate_attachments_are_limited(service):
client, sender, _ = service
base = [("submission_id", (None, ID)), ("type", (None, "problem")),
("message", (None, "Complaint"))]
response = client.post("/v1/feedback", files=base + [
("attachments", (f"{i}.png", PNG + b"x" * (7 * 1024 * 1024), "image/png"))
for i in range(3)])
assert response.status_code == 413
assert not sender.messages
def test_chunked_body_without_content_length_is_limited(service):
client, sender, _ = service
events, output = iter([
{"type": "http.request", "body": b"x" * (11 * 1024 * 1024), "more_body": True},
{"type": "http.request", "body": b"x" * (11 * 1024 * 1024), "more_body": False},
]), []
async def receive():
return next(events)
async def send(event):
output.append(event)
scope = {"type": "http", "asgi": {"version": "3.0"}, "method": "POST",
"path": "/v1/feedback", "raw_path": b"/v1/feedback", "query_string": b"",
"headers": [(b"content-type", b"multipart/form-data; boundary=abc")],
"client": ("127.0.0.1", 123), "server": ("localhost", 80), "scheme": "http"}
asyncio.run(client.app(scope, receive, send))
assert output[0]["status"] == 413
assert not sender.messages
def test_oversized_declared_body_is_rejected_before_parsing(service):
client, sender, _ = service
response = client.post("/v1/feedback", content=b"", headers={
"content-type": "multipart/form-data; boundary=abc", "content-length": str(100 * 1024 * 1024)})
assert response.status_code == 413
assert not sender.messages
def test_request_rate_is_limited_before_smtp(service):
_, sender, settings = service
settings.ip_requests_per_hour = 2
with TestClient(create_app(settings, sender)) as client:
assert post(client).status_code == 200
assert post(client).status_code == 200
response = post(client)
assert response.status_code == 429
assert response.headers["retry-after"]
assert len(sender.messages) == 1
def test_unsupported_body_format_cannot_use_unbounded_form_parser(service):
client, sender, _ = service
assert client.post("/v1/feedback", data={"type": "problem"}).status_code == 415
assert not sender.messages
def test_persistent_metadata_does_not_contain_personal_text(service):
client, _, settings = service
assert post(client, message="private complaint", contact="private@example.com").status_code == 200
with sqlite3.connect(settings.database_path) as db:
values = repr(db.execute("SELECT * FROM submissions").fetchall())
assert "private complaint" not in values
assert "private@example.com" not in values
def test_invalid_environment_addresses_fail_before_startup(tmp_path):
for sender in ["feedback@example.com\r\nBcc: victim@example.com", "Display Name <a@example.com>", ""]:
settings = Settings(database_path=str(tmp_path / "db"), smtp_username="user", smtp_password="secret",
mail_from=sender, mail_to="owner@example.com")
with pytest.raises(ValueError):
create_app(settings, RecordingSender())
def test_global_rate_cannot_be_bypassed_by_changing_client_ips(service):
_, sender, settings = service
settings.global_requests_per_hour = 2
app = create_app(settings, sender)
for ip, expected in [("1.2.3.4", 200), ("2.3.4.5", 200), ("3.4.5.6", 429)]:
with TestClient(app, client=(ip, 123)) as client:
assert post(client).status_code == expected
assert len(sender.messages) == 1
def test_concurrency_is_bounded_before_parsing(service):
_, sender, settings = service
settings.max_concurrent_requests = 1
started, finish = threading.Event(), threading.Event()
def blocking_sender(message):
started.set()
assert finish.wait(10)
sender(message)
with TestClient(create_app(settings, blocking_sender)) as client:
result = []
worker = threading.Thread(target=lambda: result.append(post(client)))
worker.start()
assert started.wait(10)
try:
response = post(client)
assert response.status_code == 503
assert response.json()["error"] == "server_busy"
finally:
finish.set()
worker.join(10)
assert result[0].status_code == 200
assert len(sender.messages) == 1
def test_lost_database_commit_after_acceptance_never_claims_success(service, monkeypatch):
client, sender, _ = service
def storage_failure(self, submission_id, status):
raise sqlite3.OperationalError("simulated storage failure")
monkeypatch.setattr(SubmissionStore, "finish", storage_failure)
response = post(client)
assert response.status_code == 503
assert response.json()["success"] is False
assert response.json()["error"] == "storage_unavailable"
assert post(client).status_code == 409
assert len(sender.messages) == 1
class ScriptedSmtp:
"""Protocol boundary fake; no socket is opened and no real mail is sent."""
def __init__(self, phase="accept"):
self.phase, self.events, self.context = phase, [], None
def login(self, user, password):
self.events.append("login")
if self.phase == "auth_fail":
raise smtplib.SMTPAuthenticationError(535, b"authentication rejected")
def ehlo(self):
self.events.append("ehlo")
def starttls(self, context):
self.context = context
self.events.append("starttls")
def mail(self, sender):
self.events.append(("mail", sender))
return 250, b"ok"
def rcpt(self, recipient):
self.events.append(("rcpt", recipient))
return (550, b"rejected") if self.phase == "recipient_fail" else (250, b"ok")
def data(self, payload):
self.events.append("data")
if self.phase == "lost_ack":
raise smtplib.SMTPServerDisconnected("connection lost")
if self.phase == "data_reject":
raise smtplib.SMTPDataError(552, b"too big")
return 250, b"accepted"
def quit(self):
self.events.append("quit")
if self.phase == "quit_fail":
raise smtplib.SMTPServerDisconnected("quit lost")
def close(self):
self.events.append("close")
@pytest.mark.parametrize("phase, expected", [
("accept", None), ("quit_fail", None), ("auth_fail", SendFailure),
("recipient_fail", SendFailure), ("data_reject", SendFailure), ("lost_ack", UncertainFailure),
])
def test_smtp_outcome_tracks_data_acknowledgement(service, monkeypatch, phase, expected):
client, sender, settings = service
assert post(client).status_code == 200
fake = ScriptedSmtp(phase)
def connect(host, port, timeout, context):
assert host == "smtp.feishu.cn" and port == 465
assert context.check_hostname and context.verify_mode == 2
return fake
monkeypatch.setattr(smtplib, "SMTP_SSL", connect)
if expected:
with pytest.raises(expected):
SmtpSender(settings)(sender.messages[0])
else:
SmtpSender(settings)(sender.messages[0])
assert ("mail", "feedback@example.com") in fake.events or phase == "auth_fail"
assert ("rcpt", "owner@example.com") in fake.events or phase == "auth_fail"
def test_starttls_is_verified_before_authentication(service, monkeypatch):
client, sender, settings = service
assert post(client).status_code == 200
settings.smtp_port, settings.smtp_tls = 587, "starttls"
fake = ScriptedSmtp()
monkeypatch.setattr(smtplib, "SMTP", lambda host, port, timeout: fake)
SmtpSender(settings)(sender.messages[0])
assert fake.events[:4] == ["ehlo", "starttls", "ehlo", "login"]
assert fake.context.check_hostname and fake.context.verify_mode == 2
@pytest.mark.parametrize("name, mime, data", [
("a.jpg", "image/jpeg", b"\xff\xd8\xff\xe0fixture"),
("a.gif", "image/gif", b"GIF89afixture"),
("a.webp", "image/webp", b"RIFF\x00\x00\x00\x00WEBPfixture"),
("a.heic", "image/heic", b"\x00\x00\x00\x14ftypheic\x00\x00\x00\x00mif1"),
("a.mp4", "video/mp4", b"\x00\x00\x00\x14ftypisom\x00\x00\x00\x00mp42"),
("a.m4v", "video/x-m4v", b"\x00\x00\x00\x14ftypM4V \x00\x00\x00\x00mp42"),
("a.mov", "video/quicktime", b"\x00\x00\x00\x14ftypqt \x00\x00\x00\x00qt "),
("a.webm", "video/webm", b"\x1a\x45\xdf\xa3doctypewebm"),
])
def test_supported_media_signatures_reach_mail_attachment(service, name, mime, data):
client, sender, _ = service
response = client.post("/v1/feedback", files=[
("submission_id", (None, ID)), ("type", (None, "problem")), ("message", (None, "Issue")),
("attachments", (name, data, mime)),
])
assert response.status_code == 200
part = list(sender.messages[0].iter_attachments())[0]
assert part.get_payload(decode=True) == data
assert part.get_content_type() == mime
def test_expired_metadata_is_removed_at_startup_without_a_new_submission(service):
client, sender, settings = service
assert post(client).status_code == 200
with sqlite3.connect(settings.database_path) as db:
db.execute("UPDATE submissions SET updated=?", (time.time() - 8 * 86400,))
create_app(settings, sender)
with sqlite3.connect(settings.database_path) as db:
assert db.execute("SELECT COUNT(*) FROM submissions").fetchone()[0] == 0
def test_slow_chunked_upload_times_out_before_sending(service):
_, sender, settings = service
settings.upload_timeout_seconds = 0.01
app = create_app(settings, sender)
output = []
async def receive():
await asyncio.sleep(0.1)
return {"type": "http.request", "body": b"", "more_body": True}
async def send(event):
output.append(event)
scope = {"type": "http", "asgi": {"version": "3.0"}, "method": "POST",
"path": "/v1/feedback", "raw_path": b"/v1/feedback", "query_string": b"",
"headers": [(b"content-type", b"multipart/form-data; boundary=abc")],
"client": ("127.0.0.1", 123), "server": ("localhost", 80), "scheme": "http"}
asyncio.run(app(scope, receive, send))
assert output[0]["status"] == 408
assert not sender.messages
def test_attachment_filename_extension_matches_detected_type(service):
client, sender, _ = service
response = client.post("/v1/feedback", files=[
("submission_id", (None, ID)), ("type", (None, "problem")), ("message", (None, "Issue")),
("attachments", ("misleading.mov", PNG, "image/png")),
])
assert response.status_code == 200
assert list(sender.messages[0].iter_attachments())[0].get_filename() == "misleading.png"
def test_absolute_upload_deadline_applies_even_when_chunks_are_ready(service, monkeypatch):
_, sender, settings = service
settings.upload_timeout_seconds = 1
app = create_app(settings, sender)
clock = iter([0, 0, 0, 2])
# Replace only this module's clock so the asyncio loop keeps real monotonic time.
monkeypatch.setattr(feedback_service, "time", SimpleNamespace(
monotonic=lambda: next(clock, 2), time=time.time))
events = iter([
{"type": "http.request", "body": b"first chunk", "more_body": True},
{"type": "http.request", "body": b"second chunk", "more_body": False},
])
output = []
async def receive():
return next(events)
async def send(event):
output.append(event)
scope = {"type": "http", "asgi": {"version": "3.0"}, "method": "POST",
"path": "/v1/feedback", "raw_path": b"/v1/feedback", "query_string": b"",
"headers": [(b"content-type", b"multipart/form-data; boundary=abc")],
"client": ("127.0.0.1", 123), "server": ("localhost", 80), "scheme": "http"}
asyncio.run(app(scope, receive, send))
assert output[0]["status"] == 408
assert not sender.messages
+78
View File
@@ -0,0 +1,78 @@
# 反馈邮件服务:1Panel 部署说明
已准备好服务文件,尚未连接服务器、部署或发送真实邮件。
| 项目 | 配置 |
|---|---|
| 服务器 | Ubuntu / 1Panel,公网 IP `43.138.57.216` |
| 域名 | `app.glowtps.cn` |
| App 提交地址 | `https://app.glowtps.cn/v1/feedback` |
| 飞书发件邮箱 | `authors@glowtps.cn` |
| 固定收件邮箱 | `tps-david@glowtps.cn` |
| 发信服务器 | `smtp.feishu.cn`,端口 `465`,SSL |
## 1. 上传服务文件
在 1Panel 文件管理中,准备 `/opt/app-feedback` 目录,把本目录的服务文件上传进去。需要保留 `feedback_service.py`、`main.py`、`requirements.txt`、`Dockerfile`、`docker-compose.yml`、`.dockerignore`、`.env.example` 和 `reverse-proxy.conf.example`。
不要上传开发电脑上的 `.deps`、`.venv`、`.test*`、缓存目录、数据库或已有的 `.env`。这些文件不属于部署包。
## 2. 在服务器填写 `.env`
将 `.env.example` 复制为 `.env`,确认以下公开配置:
```dotenv
SMTP_HOST=smtp.feishu.cn
SMTP_PORT=465
SMTP_TLS=ssl
SMTP_USERNAME=authors@glowtps.cn
MAIL_FROM=authors@glowtps.cn
MAIL_TO=tps-david@glowtps.cn
DATABASE_PATH=/app/data/feedback.sqlite3
```
另外填写两个服务器专用秘密:
- `SMTP_PASSWORD`:在飞书邮箱启用第三方邮件客户端访问后取得的客户端授权密码。仅在服务器 `.env` 中填写,不要写进 App、APK 或源代码,也不要发到聊天中。
- `IDEMPOTENCY_SECRET`:用于保护重复提交记录的随机密钥,至少 32 字节。可在服务器终端执行 `openssl rand -hex 32` 生成,再填入 `.env`。后续更新服务时保留这个值。
两个示例占位值必须替换。建议用英文单引号包住秘密值,避免其中的 `$` 被 Compose 替换。通过文件管理或 `chmod 600 .env` 限制 `.env` 的读取权限。其余限制参数可以保留默认值。
## 3. 启动 Compose
在 1Panel 的服务器终端进入上传目录并启动:
```sh
cd /opt/app-feedback
docker compose up -d --build
docker compose ps
curl --fail http://127.0.0.1:18080/healthz
```
也可以使用 1Panel 的容器编排入口导入 `docker-compose.yml`;需要确认构建目录是 `/opt/app-feedback`,同一目录内能找到 Dockerfile、服务文件和 `.env`。
`healthz` 正常时会返回 `{"status":"ok"}`。它不会发送邮件,也不代表飞书密码已经验证。如果启动失败,在 1Panel 查看容器日志。不要公开分享 `.env`、完整容器详情或 `docker compose config` 输出,这些内容可能包含密码。
## 4. 配置域名、HTTPS 和反向代理
1. 在域名 DNS 中,将 `app.glowtps.cn` 的 A 记录指向 `43.138.57.216`。
2. 在腾讯云安全组中允许网站所需的 80、443 端口;`18080` 保持对公网关闭。
3. 在 1Panel 配置 `app.glowtps.cn` 网站,申请或选择 HTTPS 证书,启用 HTTPS 跳转。
4. 设置反向代理到 `http://127.0.0.1:18080`,并参考 `reverse-proxy.conf.example` 加入反馈接口和健康检查的配置。保留 1Panel 生成的证书设置;如果域名已有网站,只增加对应接口,保留其他页面和路由。
5. 通过 1Panel 检查配置后再重新加载 OpenResty,访问 `https://app.glowtps.cn/healthz` 验证。
**网络方式需要确认:**上述地址要求 OpenResty 能访问宿主机的 `127.0.0.1`。如果 OpenResty 使用独立的 Docker 桥接网络,它的 `127.0.0.1` 是自己的容器;此时需让 OpenResty 和反馈服务加入同一个私有 Docker 网络,改为代理 `feedback:8000`,并取消宿主机端口发布、调整可信代理设置。不要通过向公网开放 `18080` 来解决连接问题。具体配置见同目录英文 README。
## 5. 由操作者进行一次真实邮件验证
服务器和证书准备完成后,由操作者从 App 提交一条明确标记为“部署测试”的反馈,确认接口返回 `success:true`、`status:sent`,并在 `tps-david@glowtps.cn` 检查收件。再使用**相同提交 ID 和相同内容**重试一次,确认没有重复邮件。
接口成功表示飞书发信服务器已接受邮件;是否进入收件箱仍需检查。若返回“发送状态不确定”,先检查收件和投递记录,不要换一个提交 ID 强行重发。完整错误说明见 README。
## 6. 更新时保留数据和秘密
服务使用 Compose 的 SQLite 数据卷保存重复提交状态,默认保留七天;文本、联系方式和附件不写入这个数据库。邮件本身按飞书及收件邮箱的策略保存。
更新时保留 `.env`、原来的 `IDEMPOTENCY_SECRET` 和 SQLite 数据卷,继续在同一部署目录使用同一个 Compose 项目。需要停服时使用 `docker compose down`,**不要使用 `docker compose down -v`**,后者会删除数据卷并丢失重复提交记录。需要备份时采用一致的 SQLite 备份方式,详细方法见 README。
目前还需要在服务器完成:填写飞书客户端授权密码、生成持久密钥、启动容器、检查 DNS/证书/代理网络,并执行一次受控的真实收件验证。