alex/ASE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

aggiunto server sftp con variabile d'ambiente FTP_MODE

Browse Source
This commit is contained in:
alex
2025-11-02 16:19:24 +01:00
parent e0f95919be
commit 10d58a3124
6 changed files with 570 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
.env.example
View File

@@ -2,7 +2,17 @@
# Copia questo file in .env e modifica i valori secondo le tue necessità
# ============================================================================
# FTP Server Configuration
# Server Mode Configuration
# ============================================================================
# Server protocol mode: ftp or sftp
# - ftp: Traditional FTP server (requires FTP_PASSIVE_PORTS and FTP_EXTERNAL_IP)
# - sftp: SFTP server over SSH (more secure, requires SSH host key)
# Default: ftp
FTP_MODE=ftp
# ============================================================================
# FTP Server Configuration (only for FTP_MODE=ftp)
# ============================================================================
# Porta iniziale del range di porte passive FTP

54
docker-compose.example.yml
View File

@@ -1,15 +1,21 @@
version: '3.8'
services:
# ============================================================================
# FTP Server (Traditional FTP)
# ============================================================================
ftp-server:
build: .
container_name: ase-ftp-server
ports:
- "2121:2121" # FTP control port
- "60000-60099:60000-60099" # FTP passive ports range
- "40000-40449:40000-40449" # FTP passive ports range
environment:
# Server Mode
FTP_MODE: "ftp" # Mode: ftp or sftp
# FTP Configuration
FTP_PASSIVE_PORTS: "60000" # Prima porta del range passivo
FTP_PASSIVE_PORTS: "40000" # Prima porta del range passivo
FTP_EXTERNAL_IP: "192.168.1.100" # IP esterno/VIP da pubblicizzare ai client
# Database Configuration
@@ -22,7 +28,7 @@ services:
# Logging (opzionale)
LOG_LEVEL: "INFO"
volumes:
- ./logs:/app/logs
- ./logs/ftp:/app/logs
- ./data:/app/data
depends_on:
- mysql-server
@@ -30,15 +36,49 @@ services:
networks:
- ase-network
# Esempio di setup HA con più istanze FTP
# ============================================================================
# SFTP Server (SSH File Transfer Protocol)
# ============================================================================
sftp-server:
build: .
container_name: ase-sftp-server
ports:
- "2222:22" # SFTP port (SSH)
environment:
# Server Mode
FTP_MODE: "sftp" # Mode: ftp or sftp
# Database Configuration
DB_HOST: "mysql-server"
DB_PORT: "3306"
DB_USER: "ase_user"
DB_PASSWORD: "your_secure_password"
DB_NAME: "ase_lar"
# Logging (opzionale)
LOG_LEVEL: "INFO"
volumes:
- ./logs/sftp:/app/logs
- ./data:/app/data
- ./ssh_host_key:/app/ssh_host_key:ro # SSH host key (generate with: ssh-keygen -t rsa -f ssh_host_key)
depends_on:
- mysql-server
restart: unless-stopped
networks:
- ase-network
# ============================================================================
# Esempio: Setup HA con più istanze FTP (stesso VIP)
# ============================================================================
ftp-server-2:
build: .
container_name: ase-ftp-server-2
ports:
- "2122:2121" # Diversa porta di controllo per seconda istanza
- "61000-61099:60000-60099" # Diverso range passivo
- "41000-41449:40000-40449" # Diverso range passivo sull'host
environment:
FTP_PASSIVE_PORTS: "60000" # Stessa config, ma mappata su porte diverse dell'host
FTP_MODE: "ftp"
FTP_PASSIVE_PORTS: "40000" # Stessa config interna
FTP_EXTERNAL_IP: "192.168.1.100" # Stesso VIP condiviso
DB_HOST: "mysql-server"
DB_PORT: "3306"
@@ -47,7 +87,7 @@ services:
DB_NAME: "ase_lar"
LOG_LEVEL: "INFO"
volumes:
- ./logs2:/app/logs
- ./logs/ftp2:/app/logs
- ./data:/app/data
depends_on:
- mysql-server

252
docs/FTP_SFTP_SETUP.md Normal file
View File

@@ -0,0 +1,252 @@
# FTP/SFTP Server Setup Guide
Il sistema ASE supporta sia FTP che SFTP utilizzando lo stesso codice Python. La modalità viene selezionata tramite la variabile d'ambiente `FTP_MODE`.
## Modalità Supportate
### FTP (File Transfer Protocol)
- **Protocollo**: FTP classico
- **Porta**: 21 (o configurabile)
- **Sicurezza**: Non criptato (considera FTPS per produzione)
- **Porte passive**: Richiede un range di porte configurabile
- **Caso d'uso**: Compatibilità con client legacy, performance
### SFTP (SSH File Transfer Protocol)
- **Protocollo**: SSH-based file transfer
- **Porta**: 22 (o configurabile)
- **Sicurezza**: Criptato tramite SSH
- **Porte passive**: Non necessarie (usa solo la porta SSH)
- **Caso d'uso**: Sicurezza, firewall-friendly
## Configurazione
### Variabili d'Ambiente
#### Comuni a entrambi i protocolli
```bash
FTP_MODE=ftp # o "sftp"
DB_HOST=mysql-server
DB_PORT=3306
DB_USER=ase_user
DB_PASSWORD=password
DB_NAME=ase_lar
LOG_LEVEL=INFO
```
#### Specifiche per FTP
```bash
FTP_PASSIVE_PORTS=40000 # Prima porta del range passivo
FTP_EXTERNAL_IP=192.168.1.100 # VIP per HA
```
#### Specifiche per SFTP
```bash
# Nessuna variabile specifica - richiede solo SSH host key
```
## Setup Docker Compose
### Server FTP
```yaml
services:
ftp-server:
build: .
container_name: ase-ftp-server
ports:
- "2121:2121"
- "40000-40449:40000-40449"
environment:
FTP_MODE: "ftp"
FTP_PASSIVE_PORTS: "40000"
FTP_EXTERNAL_IP: "192.168.1.100"
DB_HOST: "mysql-server"
DB_USER: "ase_user"
DB_PASSWORD: "password"
DB_NAME: "ase_lar"
volumes:
- ./logs/ftp:/app/logs
- ./data:/app/data
```
### Server SFTP
```yaml
services:
sftp-server:
build: .
container_name: ase-sftp-server
ports:
- "2222:22"
environment:
FTP_MODE: "sftp"
DB_HOST: "mysql-server"
DB_USER: "ase_user"
DB_PASSWORD: "password"
DB_NAME: "ase_lar"
volumes:
- ./logs/sftp:/app/logs
- ./data:/app/data
- ./ssh_host_key:/app/ssh_host_key:ro
```
## Generazione SSH Host Key per SFTP
Prima di avviare il server SFTP, genera la chiave SSH:
```bash
ssh-keygen -t rsa -b 4096 -f ssh_host_key -N ""
```
Questo crea:
- `ssh_host_key` - Chiave privata (monta nel container)
- `ssh_host_key.pub` - Chiave pubblica
## Autenticazione
Entrambi i protocolli usano lo stesso sistema di autenticazione:
1. **Admin user**: Configurato in `ftp.ini`
2. **Virtual users**: Salvati nella tabella `virtusers` del database
3. **Password**: SHA256 hash
4. **Sincronizzazione**: Automatica tra tutte le istanze (legge sempre dal DB)
## Comandi SITE (solo FTP)
I comandi SITE sono disponibili solo in modalità FTP:
```bash
ftp> site addu username password # Aggiungi utente
ftp> site disu username # Disabilita utente
ftp> site enau username # Abilita utente
ftp> site lstu # Lista utenti
```
In modalità SFTP, usa lo script `load_ftp_users.py` per gestire gli utenti.
## High Availability (HA)
### Setup HA con FTP
Puoi eseguire più istanze FTP che condividono lo stesso VIP:
```yaml
ftp-server-1:
environment:
FTP_EXTERNAL_IP: "192.168.1.100" # VIP condiviso
ports:
- "2121:2121"
- "40000-40449:40000-40449"
ftp-server-2:
environment:
FTP_EXTERNAL_IP: "192.168.1.100" # Stesso VIP
ports:
- "2122:2121"
- "41000-41449:40000-40449" # Range diverso sull'host
```
### Setup HA con SFTP
Più semplice, nessuna configurazione di porte passive:
```yaml
sftp-server-1:
ports:
- "2222:22"
sftp-server-2:
ports:
- "2223:22"
```
## Testing
### Test FTP
```bash
ftp 192.168.1.100 2121
# Username: admin (o utente dal database)
# Password: <password>
ftp> ls
ftp> put file.csv
ftp> by
```
### Test SFTP
```bash
sftp -P 2222 admin@192.168.1.100
# Password: <password>
sftp> ls
sftp> put file.csv
sftp> exit
```
## Monitoring
I log sono disponibili sia su file che su console (Docker):
```bash
# Visualizza log FTP
docker logs ase-ftp-server
# Visualizza log SFTP
docker logs ase-sftp-server
# Segui i log in tempo reale
docker logs -f ase-ftp-server
```
## Troubleshooting
### FTP: Errore "Can't connect to passive port"
- Verifica che il range di porte passive sia mappato correttamente in Docker
- Controlla che `FTP_EXTERNAL_IP` sia impostato correttamente
- Verifica che `FTP_PASSIVE_PORTS` corrisponda al range configurato
### SFTP: Errore "Connection refused"
- Verifica che l'SSH host key esista e sia montato correttamente
- Controlla i permessi del file SSH host key (deve essere leggibile)
- Installa `asyncssh`: `pip install asyncssh`
### Autenticazione fallita (entrambi)
- Verifica che il database sia raggiungibile
- Controlla che le credenziali del database siano corrette
- Verifica che l'utente esista nella tabella `virtusers` e sia abilitato (`disabled_at IS NULL`)
## Dipendenze
### FTP
```bash
pip install pyftpdlib mysql-connector-python
```
### SFTP
```bash
pip install asyncssh aiomysql
```
## Performance
- **FTP**: Più veloce per trasferimenti di file grandi, minore overhead
- **SFTP**: Leggermente più lento a causa della crittografia SSH, ma più sicuro
## Sicurezza
### FTP
- ⚠️ Non criptato - considera FTPS per produzione
- Abilita `permit_foreign_addresses` per NAT/proxy
- Usa firewall per limitare accesso
### SFTP
- ✅ Completamente criptato tramite SSH
- ✅ Più sicuro per Internet pubblico
- ✅ Supporta autenticazione a chiave pubblica (future enhancement)
## Migration
Per migrare da FTP a SFTP:
1. Avvia server SFTP con stesse credenziali database
2. Testa connessione SFTP
3. Migra client gradualmente
4. Spegni server FTP quando tutti i client sono migrati
Gli utenti e i dati rimangono gli stessi!

3
pyproject.toml
View File

@@ -7,13 +7,14 @@ requires-python = ">=3.12"
dependencies = [
"aiomysql>=0.2.0",
"cryptography>=45.0.3",
"mysql-connector-python>=9.3.0", # Needed for synchronous DB connections (ftp_csv_receiver.py, load_ftp_users.py)
"mysql-connector-python>=9.3.0", # Needed for synchronous DB connections (ftp_csv_receiver.py, load_ftp_users.py)
"pyftpdlib>=2.0.1",
"pyproj>=3.7.1",
"utm>=0.8.1",
"aiofiles>=24.1.0",
"aiosmtplib>=3.0.2",
"aioftp>=0.22.3",
"asyncssh>=2.21.1",
]
[dependency-groups]

73
src/ftp_csv_receiver.py
View File

@@ -1,11 +1,17 @@
#!.venv/bin/python
"""
This module implements an FTP server with custom commands for
This module implements an FTP/SFTP server with custom commands for
managing virtual users and handling CSV file uploads.
Server mode is controlled by FTP_MODE environment variable:
- FTP_MODE=ftp (default): Traditional FTP server
- FTP_MODE=sftp: SFTP (SSH File Transfer Protocol) server
"""
import asyncio
import logging
import os
import sys
from hashlib import sha256
from logging.handlers import RotatingFileHandler
from pathlib import Path
@@ -140,15 +146,9 @@ def setup_logging(log_filename: str):
root_logger.info("Logging FTP configurato con rotation (10MB, 5 backup) e console output")
def main():
"""Main function to start the FTP server."""
# Load the configuration settings
cfg = setting.Config()
def start_ftp_server(cfg):
"""Start traditional FTP server."""
try:
# Configure logging first
setup_logging(cfg.logfilename)
# Initialize the authorizer with database support
# This authorizer checks the database on every login, ensuring
# all FTP server instances stay synchronized without restarts
@@ -185,7 +185,60 @@ def main():
server.serve_forever()
except Exception as e:
logger.error("Exit with error: %s.", e)
logger.error("FTP server error: %s", e, exc_info=True)
sys.exit(1)
async def start_sftp_server_async(cfg):
"""Start SFTP server (async)."""
try:
from utils.servers.sftp_server import start_sftp_server
logger.info(f"Starting SFTP server on port {cfg.service_port}")
logger.info(f"Database connection: {cfg.dbuser}@{cfg.dbhost}:{cfg.dbport}/{cfg.dbname}")
# Start SFTP server
server = await start_sftp_server(cfg, host="0.0.0.0", port=cfg.service_port)
# Keep server running
await asyncio.Event().wait()
except ImportError as e:
logger.error("SFTP mode requires 'asyncssh' library. Install with: pip install asyncssh")
logger.error(f"Error: {e}")
sys.exit(1)
except Exception as e:
logger.error("SFTP server error: %s", e, exc_info=True)
sys.exit(1)
def main():
"""Main function to start FTP or SFTP server based on FTP_MODE environment variable."""
# Load the configuration settings
cfg = setting.Config()
# Configure logging first
setup_logging(cfg.logfilename)
# Get server mode from environment variable (default: ftp)
server_mode = os.getenv("FTP_MODE", "ftp").lower()
if server_mode not in ["ftp", "sftp"]:
logger.error(f"Invalid FTP_MODE: {server_mode}. Valid values: ftp, sftp")
sys.exit(1)
logger.info(f"Server mode: {server_mode.upper()}")
try:
if server_mode == "ftp":
start_ftp_server(cfg)
elif server_mode == "sftp":
asyncio.run(start_sftp_server_async(cfg))
except KeyboardInterrupt:
logger.info("Server stopped by user")
except Exception as e:
logger.error("Unexpected error: %s", e, exc_info=True)
sys.exit(1)
if __name__ == "__main__":

194
src/utils/servers/sftp_server.py Normal file
View File

@@ -0,0 +1,194 @@
"""
SFTP Server implementation using asyncssh.
Shares the same authentication system and file handling logic as the FTP server.
"""
import asyncio
import logging
from pathlib import Path
import asyncssh
from utils.connect import file_management
from utils.database.connection import connetti_db_async
logger = logging.getLogger(__name__)
class ASESFTPServer(asyncssh.SFTPServer):
"""Custom SFTP server that handles file uploads with the same logic as FTP server."""
def __init__(self, chan):
"""Initialize SFTP server with channel."""
super().__init__(chan)
# Get config from connection (set during authentication)
self.cfg = chan.get_connection()._cfg
async def close(self):
"""Called when SFTP session is closed."""
logger.info(f"SFTP session closed for user: {self._chan.get_connection().get_extra_info('username')}")
await super().close()
class ASESSHServer(asyncssh.SSHServer):
"""Custom SSH server for SFTP authentication using database."""
def __init__(self, cfg):
"""Initialize SSH server with configuration."""
self.cfg = cfg
super().__init__()
def connection_made(self, conn):
"""Called when connection is established."""
# Store config in connection for later use
conn._cfg = self.cfg
logger.info(f"SSH connection from {conn.get_extra_info('peername')[0]}")
def connection_lost(self, exc):
"""Called when connection is lost."""
if exc:
logger.error(f"SSH connection lost: {exc}")
async def validate_password(self, username, password):
"""
Validate user credentials against database.
Same logic as DatabaseAuthorizer for FTP.
"""
from hashlib import sha256
# Hash the provided password
password_hash = sha256(password.encode("UTF-8")).hexdigest()
# Check if user is admin
if username == self.cfg.adminuser[0]:
if self.cfg.adminuser[1] == password_hash:
logger.info(f"Admin user '{username}' authenticated successfully")
return True
else:
logger.warning(f"Failed admin login attempt for user: {username}")
return False
# For regular users, check database
try:
conn = await connetti_db_async(self.cfg)
cur = await conn.cursor()
# Query user from database
await cur.execute(
f"SELECT ftpuser, hash, virtpath, perm, disabled_at FROM {self.cfg.dbname}.{self.cfg.dbusertable} WHERE ftpuser = %s",
(username,)
)
result = await cur.fetchone()
await cur.close()
conn.close()
if not result:
logger.warning(f"SFTP login attempt for non-existent user: {username}")
return False
ftpuser, stored_hash, virtpath, perm, disabled_at = result
# Check if user is disabled
if disabled_at is not None:
logger.warning(f"SFTP login attempt for disabled user: {username}")
return False
# Verify password
if stored_hash != password_hash:
logger.warning(f"Invalid password for SFTP user: {username}")
return False
# Authentication successful - ensure user directory exists
try:
Path(virtpath).mkdir(parents=True, exist_ok=True)
except Exception as e:
logger.error(f"Failed to create directory for user {username}: {e}")
return False
logger.info(f"Successful SFTP login for user: {username}")
return True
except Exception as e:
logger.error(f"Database error during SFTP authentication for user {username}: {e}", exc_info=True)
return False
def password_auth_supported(self):
"""Enable password authentication."""
return True
def begin_auth(self, username):
"""Called when authentication begins."""
logger.debug(f"Authentication attempt for user: {username}")
return True
class SFTPFileHandler(asyncssh.SFTPServer):
"""Extended SFTP server with file upload handling."""
def __init__(self, chan):
super().__init__(chan)
self.cfg = chan.get_connection()._cfg
async def close(self):
"""Handle session close."""
await super().close()
# Override file operations to add custom handling
async def rename(self, oldpath, newpath):
"""
Handle file rename/move - called when upload completes.
This is where we trigger the CSV processing like in FTP.
"""
result = await super().rename(oldpath, newpath)
# Check if it's a CSV file that was uploaded
if newpath.lower().endswith('.csv'):
try:
# Trigger file processing (same as FTP on_file_received)
logger.info(f"CSV file uploaded via SFTP: {newpath}")
# Create a mock handler object with required attributes
mock_handler = type('obj', (object,), {
'cfg': self.cfg,
'username': self._chan.get_connection().get_extra_info('username')
})()
# Call the same file_management function used by FTP
file_management.on_file_received(mock_handler, newpath)
except Exception as e:
logger.error(f"Error processing SFTP uploaded file {newpath}: {e}", exc_info=True)
return result
async def start_sftp_server(cfg, host='0.0.0.0', port=22):
"""
Start SFTP server.
Args:
cfg: Configuration object
host: Host to bind to
port: Port to bind to
Returns:
asyncssh server object
"""
logger.info(f"Starting SFTP server on {host}:{port}")
# Create SSH server
ssh_server = ASESSHServer(cfg)
# Start asyncssh server
server = await asyncssh.create_server(
lambda: ssh_server,
host,
port,
server_host_keys=['/app/ssh_host_key'], # You'll need to generate this
sftp_factory=SFTPFileHandler,
session_encoding=None, # Binary mode for file transfers
)
logger.info(f"SFTP server started successfully on {host}:{port}")
logger.info(f"Database connection: {cfg.dbuser}@{cfg.dbhost}:{cfg.dbport}/{cfg.dbname}")
return server