javohir.dev
KOENUZ
All projectsLive

Smart Farm

Real-time greenhouse IoT monitoring and automatic irrigation that closes the loop from ESP32 sensors through MQTT, a pipeline and rules to the pump and back to the device

Open live service: farm.javohir.devGitHub code

What this project proves

  • Proving with measurements that IoT data is not lost: 24/24 through a 132 s MongoDB outage and 7/7 through a 20 s backend stop; the losses on the old code (1 of 12, 1 per deploy) were measured first
  • A security audit and fixes: more than 12 issues, including ADMIN escalation at sign-up, arbitrary field writes, cross-member access, unlimited OTP guessing and a hard-coded JWT key, closed after first checking production for abuse
  • Closing the loop between device and backend: RELAY_ON/OFF goes out over MQTT, pump run time is capped on both the server (30 min) and the firmware (1800 s), and an automation rule never fires twice at once
  • Engineering discipline: all 89 new tests were confirmed to fail on the old code, and wrong hypotheses and insufficient tests are recorded too
Role
Solo: backend, frontend, ESP32 firmware, deployment and operations (171 commits, in production since 2026-10-04)
Period
2026.01 – 2026.10
Stack
  • NestJS
  • TypeScript
  • GraphQL
  • MongoDB
  • Redis
  • BullMQ
  • MQTT
  • Socket.IO
  • React
  • Vite
  • Leaflet
  • ESP32
  • MicroPython
  • Jest
  • Docker
  • Caddy

Key numbers

ItemValueTypeEvidence
Messages saved through a 132 s MongoDB outage (DLQ 0)24 / 24MeasuredView evidence
Messages delivered through a 20 s backend stop7 / 7MeasuredView evidence
Rows saved in the 10 minutes after the IoT deploy (0 rejected)320MeasuredView evidence
Security probes rejected5 / 5MeasuredView evidence
New tests (each confirmed to fail on the old code)89From codeView evidence
Maximum pump run time (server)30 minDesign targetView evidence
Longest retry interval (30 / 60 / 120 s, DLQ after 3)120 sDesign targetView evidence
MongoDB collections40From codeView evidence

The problem

Greenhouse sensor readings drop out or spike, device clocks can be wrong, and no reading may be lost while the database or backend is briefly down. When the soil dries out the pump should start before anyone looks at a screen, and it must never keep running and flood the greenhouse because the network dropped. Nobody should ever reach someone else's farm data.

Architecture

A real ESP32 (DHT22, capacitive soil moisture, BH1750 light, water level, relay + 12 V pump) and a demo simulator in production publish over the same path to Mosquitto (TLS 8883, QoS 1). A NestJS 11 IoT pipeline checks the device API key, payload shape, topic-to-deviceId match, sensor ownership and calibration, passes through a Redis buffer (retries, DLQ), stores raw readings plus a 90-day-TTL MongoDB time series, and builds hourly, daily and monthly aggregates. It computes per-sensor Z-score anomalies (≥ 2 warning, ≥ 3 critical), a plant health index and Open-Meteo weather (10-minute cache); when an automation rule changes an actuator, RELAY_ON/OFF goes to the device over MQTT. The dashboard updates live over GraphQL (51 resolvers) and Socket.IO, in Korean, English and Uzbek.

Data model

Key constraints and indexes

  • 소유권 체인: sensor → device → greenHouse → farm → member (farms.memberId)
  • deviceApiKeys.apiKey UNIQUE, deviceId UNIQUE
  • sensor_data {sensorId:1, recordedAt:-1}, {recordedAt:-1}
  • timeSeriesSensorData: MongoDB time series, granularity seconds, TTL 90일
  • automationRules: findOneAndUpdate 필터의 cooldown 조건으로 원자적 점유
  • passwordReset / emailVerifications.attempts: 최대 5회, crypto.randomInt 6자리
  • Redis: iot:buffer:messages(hash), inflight / retry(zset), dead(list)

Key design decisions

Message buffer: a hash plus two zsets instead of a Redis list

Context
The old buffer was a single list: ack read the whole list with lrange and deleted by comparing JSON, and there were no retries at all, so messages stayed pending forever.
Decision
iot:buffer:messages (hash), iot:buffer:inflight (zset, processing start), iot:buffer:retry (zset, next attempt), iot:buffer:dead (DLQ list); ack is an O(1) hdel + zrem by id.
Trade-off
The retry logic is our own, hence 10 tests and a check against a real redis-server.

A done[] list per reading

Context
One message carries eight readings; if seven are saved and one fails, resending the whole message creates seven duplicate rows.
Decision
Successful sensors from Promise.allSettled go into done[] and are skipped on retry; retries keep the original receivedAt.
Trade-off
—

Telling authorization errors from database errors

Context
validateApiKey threw even when the database was down, and the code treated that as an invalid key and dropped the message.
Decision
Only UnauthorizedException drops the message; every other error goes to retry.
Trade-off
—

Persistent MQTT session + connecting in onApplicationBootstrap

Context
With clean:true and a new clientId every start, the broker kept nothing while the backend was down, and connecting in onModuleInit delivered queued messages before the handlers were registered, losing them.
Decision
clean:false with a stable clientId (MQTT_CLIENT_ID distinguishes a second instance), connecting in onApplicationBootstrap after every onModuleInit.
Trade-off
—

Error logging is best-effort

Context
While the database was down, handleSensorSaveError also failed writing to it, so one attempt was counted twice (DLQ after 2 attempts instead of 4).
Decision
Logging is wrapped in try/catch and only warns on failure, without touching the retry count.
Trade-off
—

Ownership checks: a global interceptor + @OwnedArgs

Context
Hand-written checks in every resolver are easy to forget in a new one.
Decision
A global OwnershipInterceptor finds the resource from the argument name and walks the chain up to farm.memberId; ambiguous arguments are mapped with @OwnedArgs({ 'input.fieldId': 'fieldMap' }).
Trade-off
Every request performs an extra ownership lookup.

Pump limit on both sides

Context
If Wi-Fi or the broker drops, RELAY_OFF may never reach the device; only the device itself can stop a flood.
Decision
The server sets autoOffAt to at most 30 minutes, and the firmware switches itself off after 1800 s + 60 s grace based on ticks_ms.
Trade-off
—

Automation rules: claiming the cooldown atomically

Context
Two sensor messages arriving together both fired the rule.
Decision
The cooldown condition sits in the findOneAndUpdate filter, so only one claims the rule.
Trade-off
—

A fallback for recordedAt

Context
ESP32s sent 1970 timestamps before NTP had synced.
Decision
If the device time is more than 24 hours old or more than 5 minutes ahead, the server uses its receive time, and second-resolution timestamps are detected automatically; the firmware retries NTP every hour.
Trade-off
—

Before and after

BeforeDB outage 58 s → 11 / 12 saved · restart → 1 lostView evidence
AfterDB outage 132 s → 24 / 24 saved, DLQ 0 · restart → 7 / 7View evidence
What changed
A hash + zset buffer with 30/60/120 s backoff, a DLQ and stuck-message recovery (e49f6d32), separating database errors from invalid keys (e49f6d32), a persistent MQTT session connecting in onApplicationBootstrap (c4010d09), and best-effort error logging (9ddb7dbb). Handler errors during a database outage went from 4 to 0. Measured in production with sf-outage-check.sh and sf-restart-check.sh under simulator load (about 2 messages per 30 s).

Incident write-up

Symptom
Sending memberRole: "ADMIN" in the sign-up mutation created an ADMIN member (probe before).
Root cause
CreateMemberInput had a memberRole field and memberModel.create(input) stored the whole input; MemberUpdateInput for profile updates also accepted role, status, email and password. The same audit found the member list open without a token ({"total":7}) because @Roles(MANAGER) had no @UseGuards(RolesGuard), and the buffer admin resolvers open because the guard ignored class-level @Roles.
Fix
Removed the field from the input and pass only allowed fields to create and $set (22a760bd); added the guard and read handler and class together with getAllAndOverride, so an invalid token gets 401.
Prevention
Before fixing I checked production for abuse (all 7 members WORKER/ACTIVE, 0 privileged, 0 suspicious calls). member-security.spec.ts has 13 tests, 12 of which fail on the old code, and after the deploy all 5 external probes are rejected.

Tests and quality

Tests
89
Coverage
—
CI
—
Test tools
Jest 30, ts-jest, 실제 redis-server, Mosquitto 2.0.18, MicroPython stub harness

Deployment and operations

Deployment
docker-compose.prod.yml on one VPS: mongo (768m), redis (128m, requirepass, AOF), mosquitto (64m, only 8883 TLS exposed, Caddy certificate read-only, allow_anonymous false, password file + ACL, persistence), backend (768m, internal network only, starts once mongo and redis are healthy), web (64m) and device-simulator (96m, demo profile). A shared Caddy handles HTTPS. Deploy: git pull --ff-only → build backend → up -d backend. Secrets (MONGO_PASSWORD, REDIS_PASSWORD, MQTT_BACKEND_PASSWORD, SECRET_TOKEN) live only in the server's .env, and compose and the backend refuse to start without them.
Backups
—
Rollback
—
Monitoring
Buffer state (pending / retrying / deadLetter) over admin GraphQL, viewing and clearing DLQ messages, and Message retry scheduled / Message moved to DLQ logs. My own ops scripts: sf-incident-check.sh, sf-security-probe-v2.ps1, sf-outage-check.sh, sf-restart-check.sh

Screenshots

IoT pipeline: ESP32 → MQTT → Redis queue and DLQ → calibration → MongoDB → Z-score → WebSocket. 54,656 readings in 24 hours, 0 pipeline errors (simulator data)
Greenhouse monitoring: weather (Open-Meteo), sensor values with optimal ranges, devices and tasks
Device status: sensor hubs, relay controller, LoRa gateway, location, last reading and soil-moisture verdict

Limits and retrospective

Current limits

  • Production data comes from the device simulator; the real ESP32 still connects to a local broker (1883, no TLS), and production needs firmware TLS and its own account
  • All devices share one smartfarm-device account (ACL sf/devices/#), so a device can write to another device's topic; the backend catches it with the API key, topic–deviceId match and sensor ownership, but the right fix is per-device accounts with an sf/devices/%u/# pattern
  • Device API keys are not hashed in the database, and one test key that landed in the repo needs rotating
  • Mosquitto reads the Caddy certificate only at startup; reloading the broker after a renewal is not automated
  • No rate limits on login and sign-up (only OTP is limited), no authentication on GraphQL subscriptions, and an unclear rule for linking social logins to existing emails
  • A single backend instance: buffer retries are claimed atomically with zrem, but the AI irrigation timer and global mode flag live in memory and could run twice on a second instance
  • Found in the audit but not fixed yet: anomaly statistics are not rolling, the monthly aggregation, setSpeed, and heartbeat flapping between online and offline
  • No CI, so tests run by hand; 15 default Nest CLI specs in account-context-module build modules without dependencies and fail regardless (unrelated to the fixes, to be cleaned up separately)
  • Gaps in the evidence: two fixes missed the production before-probe because the deploy happened first, so their before state is proven only by tests failing on the old code; outage tests ran at simulator load (about 2 messages / 30 s), so buffer size (limit 10,000) and the retry batch (50) under high load are unmeasured

Key code

Current status and next steps

An audit on 2026-10-10–11 produced nine fix commits, each with a before/after measurement. With MongoDB stopped for 132 s, 24/24 messages were saved (6 retries, DLQ 0); with the backend stopped for 20 s, 7/7 arrived, including the one sent while it was down, which came from the broker queue. On the old code a 58 s outage lost 1 of 12 messages and a deploy restart lost 1. The security audit found and closed more than 12 vulnerabilities: becoming ADMIN by sending memberRole at sign-up (22a760bd), writing arbitrary fields via profile update, reaching other members' farms, sectors and alerts (c2c5ca26), unlimited OTP guessing and Math.random OTPs, a JWT key hard-coded in the code (a5a1b62e), and an open GraphQL playground in production. Before fixing, I checked production for signs of abuse (7 members, 0 privileged accounts, 0 suspicious calls). I also fixed the backend turning the pump on only in the database without sending the device any command (3f3eacb2) and ESP32s sending 1970 timestamps before NTP sync. When my first hypothesis (errors from heartbeat) was wrong, the logs pointed to the real source (/sensors), and I recorded that a 45 s outage never exercised the error path because of the driver's 30 s wait.