Monitoring and Alerting

Checking performance at end-of-day is too late. A stuck data feed, a rejected order, or a flash crash can cause irreversible losses in minutes. Automated real-time monitoring catches problems when they are still small.

The Problem

A strategy runs in production. The data feed silently stalls at 10:15 AM - the strategy stops generating signals but nobody notices until 4 PM. By then, the portfolio has drifted and missed the day's best opportunities. Worse: a position limit was breached because a partial fill was not tracked, and the strategy doubled down. End-of-day review catches the problem 6 hours too late.

The Pattern

WRONG

# End-of-day check - too late for anything but damage assessment
def daily_report():
    pnl = portfolio.value() - portfolio.start_of_day_value()
    if pnl < -10_000:
        send_email("Bad day", f"Lost ${abs(pnl):,.0f}")
schedule.every().day.at("16:30").do(daily_report)  # 6+ hours after it started