{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# NFL weekly\n",
    "\n",
    "This page is regenerated every week by sdvplot's docs workflow. It finds the latest NFL season with play-by-play, ranks every team by EPA per play, plots offense against\n",
    "defense and lists the most efficient quarterbacks: the season to date while games are being played, the last full\n",
    "regular season in the offseason. Data: nflverse play-by-play and schedules, read through\n",
    "[sportsdataverse-py](https://py.sportsdataverse.org/)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "The season starts in September, so before then the calendar points at last season. nflverse publishes a season's\n",
    "play-by-play file with its first games; until then `load_nfl_pbp` raises `NoDataError`, and the page steps back one\n",
    "season instead of failing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime as dt\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import polars as pl\n",
    "import sportsdataverse.nfl as nfl\n",
    "from IPython.display import Markdown, display\n",
    "from sportsdataverse.errors import NoDataError\n",
    "\n",
    "import sdvplot\n",
    "\n",
    "today = dt.date.today()\n",
    "current = today.year if today.month >= 9 else today.year - 1\n",
    "\n",
    "\n",
    "def regular_season(season):\n",
    "    pbp = nfl.load_nfl_pbp([season])  # NoDataError until nflverse publishes the season\n",
    "    pbp = pbp.filter(pl.col(\"season_type\") == \"REG\")\n",
    "    if pbp.is_empty():\n",
    "        raise NoDataError(f\"no {season} regular-season plays yet\")\n",
    "    return pbp\n",
    "\n",
    "\n",
    "try:\n",
    "    season, pbp = current, regular_season(current)\n",
    "except NoDataError as err:\n",
    "    print(f\"{err}; showing {current - 1} instead\")\n",
    "    season, pbp = current - 1, regular_season(current - 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3",
   "metadata": {},
   "source": [
    "The status line below is written when the page runs. It says whether the table is the season to date or a finished\n",
    "regular season, so a stale table is never presented as current."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "schedule = nfl.load_nfl_schedule([season])\n",
    "week, n_games = pbp[\"week\"].max(), pbp[\"game_id\"].n_unique()\n",
    "unplayed = schedule.filter((pl.col(\"game_type\") == \"REG\") & pl.col(\"result\").is_null()).height\n",
    "super_bowl = schedule.filter((pl.col(\"game_type\") == \"SB\") & pl.col(\"result\").is_not_null()).height\n",
    "if unplayed:\n",
    "    status = f\"**Season to date:** the {season} season through week {week} ({n_games} games).\"\n",
    "    through = f\"through week {week}\"\n",
    "elif not super_bowl:\n",
    "    status = f\"**Playoffs:** the final {season} regular season; the playoffs are under way.\"\n",
    "    through = \"final regular season\"\n",
    "else:\n",
    "    status = f\"**Offseason:** the final {season} regular season. The {season + 1} season starts in September.\"\n",
    "    through = \"final regular season\"\n",
    "display(Markdown(status))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "## 1. Power table\n",
    "\n",
    "Every team's record and point differential from the schedule, and its EPA (expected points added) per pass or run play\n",
    "on offense and allowed on defense, from the play-by-play. Net EPA per play is offense minus defense; the last column\n",
    "compares each team's last three games with its season. `gt_merge_stack_team_color` stacks the record under the\n",
    "nickname in team colors and `gt_sdv_logos` turns the nflverse abbreviations into logos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "plays = pbp.filter(((pl.col(\"pass\") == 1) | (pl.col(\"rush\") == 1)) & pl.col(\"epa\").is_not_null())\n",
    "per_game = (\n",
    "    plays.group_by(\"game_id\", \"week\", team=\"posteam\", maintain_order=True)\n",
    "    .agg(off=pl.col(\"epa\").sum(), off_n=pl.len())\n",
    "    .join(\n",
    "        plays.group_by(\"game_id\", team=\"defteam\", maintain_order=True).agg(dfn=pl.col(\"epa\").sum(), dfn_n=pl.len()),\n",
    "        on=[\"game_id\", \"team\"],\n",
    "    )\n",
    "    .sort(\"team\", \"week\")  # a fixed row order makes every sum below come out bit-for-bit the same each week\n",
    ")\n",
    "\n",
    "\n",
    "def per_play(games):\n",
    "    return (\n",
    "        games.group_by(\"team\", maintain_order=True)\n",
    "        .agg(\n",
    "            off_epa=pl.col(\"off\").sum() / pl.col(\"off_n\").sum(),\n",
    "            def_epa=pl.col(\"dfn\").sum() / pl.col(\"dfn_n\").sum(),\n",
    "        )\n",
    "        .with_columns(net=pl.col(\"off_epa\") - pl.col(\"def_epa\"))\n",
    "    )\n",
    "\n",
    "\n",
    "last3 = per_play(per_game.group_by(\"team\", maintain_order=True).tail(3)).select(\"team\", last3=\"net\")\n",
    "\n",
    "games = schedule.filter(pl.col(\"game_type\") == \"REG\").join(pbp.select(\"game_id\").unique(), on=\"game_id\", how=\"semi\")\n",
    "sides = pl.concat(\n",
    "    [\n",
    "        games.select(team=\"home_team\", pf=\"home_score\", pa=\"away_score\"),\n",
    "        games.select(team=\"away_team\", pf=\"away_score\", pa=\"home_score\"),\n",
    "    ]\n",
    ")\n",
    "record = sides.group_by(\"team\", maintain_order=True).agg(\n",
    "    w=(pl.col(\"pf\") > pl.col(\"pa\")).sum(),\n",
    "    l=(pl.col(\"pf\") < pl.col(\"pa\")).sum(),\n",
    "    t=(pl.col(\"pf\") == pl.col(\"pa\")).sum(),\n",
    "    diff=(pl.col(\"pf\") - pl.col(\"pa\")).sum(),\n",
    ")\n",
    "record = record.with_columns(\n",
    "    record=pl.when(pl.col(\"t\") > 0).then(pl.format(\"{}-{}-{}\", \"w\", \"l\", \"t\")).otherwise(pl.format(\"{}-{}\", \"w\", \"l\"))\n",
    ")\n",
    "\n",
    "nicknames = nfl.load_nfl_teams().select(team=\"team_abbr\", name=\"team_nick\")\n",
    "power = (\n",
    "    per_play(per_game)\n",
    "    .join(last3, on=\"team\")\n",
    "    .join(record, on=\"team\")\n",
    "    .join(nicknames, on=\"team\")\n",
    "    .sort([\"net\", \"team\"], descending=[True, False])  # a tiebreaker keeps the weekly re-render stable\n",
    "    .with_row_index(\"rank\", offset=1)\n",
    "    .select(\"rank\", \"team\", \"name\", \"record\", \"diff\", \"off_epa\", \"def_epa\", \"net\", \"last3\")\n",
    ")\n",
    "power.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from great_tables import GT\n",
    "\n",
    "from sdvplot.great_tables import gt_delta, gt_merge_stack_team_color, gt_save_crop, gt_sdv_logos, gt_theme_athletic\n",
    "\n",
    "GOOD_BAD = [\"#c84630\", \"#f7f7f7\", \"#2e8b57\"]\n",
    "epa = [\"off_epa\", \"def_epa\", \"net\", \"last3\"]\n",
    "gt = (\n",
    "    GT(power, id=\"nfl-power\")  # a fixed id: great_tables otherwise draws a random one each run\n",
    "    .tab_header(f\"NFL power table, {season}\", f\"Ranked by net EPA per play, {through}\")\n",
    "    .fmt_number(epa, decimals=3, force_sign=True)\n",
    "    .fmt_number(\"diff\", decimals=0, force_sign=True)\n",
    "    .data_color(\"off_epa\", palette=GOOD_BAD, domain=[-0.3, 0.3])\n",
    "    .data_color(\"def_epa\", palette=GOOD_BAD[::-1], domain=[-0.3, 0.3])\n",
    "    .data_color(\"net\", palette=GOOD_BAD, domain=[-0.5, 0.5])\n",
    "    .tab_spanner(\"EPA per play\", epa)\n",
    "    .cols_label(\n",
    "        rank=\"\", team=\"\", name=\"Team\", diff=\"Pt diff\", off_epa=\"Offense\", def_epa=\"Defense\", net=\"Net\", last3=\"Last 3\"\n",
    "    )\n",
    "    .tab_source_note(\n",
    "        \"Data: nflverse via sportsdataverse-py. Pass and run plays; defense is EPA allowed (lower is better).\"\n",
    "    )\n",
    ")\n",
    "gt = gt_merge_stack_team_color(gt, \"name\", \"record\", \"team\", league=\"nfl\")\n",
    "gt = gt_delta(gt, \"net\", \"last3\", column_label=\"Trend\", decimals=3, arrows=True)\n",
    "gt = gt_theme_athletic(gt_sdv_logos(gt, \"team\", league=\"nfl\", height=26))\n",
    "gt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {},
   "source": [
    "`gt_save_crop` renders the same table to a trimmed PNG, ready to post."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {
    "sdvplot_gallery": {
     "alt": "NFL power table: every team's logo, record, point differential and EPA per play, ranked by net EPA.",
     "title": "NFL weekly power table"
    },
    "tags": [
     "gallery"
    ]
   },
   "outputs": [],
   "source": [
    "gt_save_crop(gt, width=900)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10",
   "metadata": {},
   "source": [
    "## 2. Offense against defense\n",
    "\n",
    "The same EPA per play as a scatter, one logo per team. The y axis is reversed so the better defenses sit higher: the\n",
    "top-right corner is where good teams live."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11",
   "metadata": {
    "sdvplot_gallery": {
     "alt": "Scatter of NFL team logos by offensive EPA per play and defensive EPA allowed per play for the current season.",
     "title": "NFL EPA per play, offense vs defense"
    },
    "tags": [
     "gallery"
    ]
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(9, 7))\n",
    "ax.scatter(power[\"off_epa\"], power[\"def_epa\"], alpha=0)  # sets the limits; the logos are the points\n",
    "ax.axvline(power[\"off_epa\"].mean(), color=\"grey\", lw=0.8, ls=\"--\")\n",
    "ax.axhline(power[\"def_epa\"].mean(), color=\"grey\", lw=0.8, ls=\"--\")\n",
    "ax.invert_yaxis()\n",
    "ax.margins(0.1)\n",
    "x0, x1 = ax.get_xlim()\n",
    "y0, y1 = ax.get_ylim()\n",
    "for x, y, text, ha in [\n",
    "    (x1, y1, \"Good offense, good defense\", \"right\"),\n",
    "    (x0, y1, \"Good defense\", \"left\"),\n",
    "    (x1, y0, \"Good offense\", \"right\"),\n",
    "    (x0, y0, \"Struggling\", \"left\"),\n",
    "]:\n",
    "    ax.text(x, y, text, ha=ha, va=\"top\" if y == y1 else \"bottom\", color=\"grey\", fontsize=9, fontstyle=\"italic\")\n",
    "sdvplot.add_logos(ax, power[\"off_epa\"], power[\"def_epa\"], power[\"team\"], league=\"nfl\", season=season, height=0.075)\n",
    "ax.set(xlabel=\"Offense: EPA per play\", ylabel=\"Defense: EPA allowed per play (reversed)\")\n",
    "ax.spines[[\"top\", \"right\"]].set_visible(False)\n",
    "ax.set_title(f\"NFL offense vs defense, {season} {through}\", loc=\"left\", fontweight=\"bold\")\n",
    "fig.text(0.99, 0.01, \"Data: nflverse via sportsdataverse-py | pass and run plays\", ha=\"right\", fontsize=8, color=\"grey\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12",
   "metadata": {},
   "source": [
    "## 3. Quarterback leaderboard\n",
    "\n",
    "EPA per dropback for every quarterback with at least 15 dropbacks per week of the season so far. nflverse's\n",
    "play-by-play carries gsis player ids, so `add_headshots(..., id_system=\"gsis\")` finds each headshot through the nflverse\n",
    "player table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {
    "sdvplot_gallery": {
     "alt": "Horizontal bars of the top NFL quarterbacks by EPA per dropback this season, with headshots and team logos.",
     "title": "NFL quarterback EPA leaderboard"
    },
    "tags": [
     "gallery"
    ]
   },
   "outputs": [],
   "source": [
    "dropbacks = pbp.filter((pl.col(\"qb_dropback\") == 1) & pl.col(\"qb_epa\").is_not_null() & pl.col(\"id\").is_not_null())\n",
    "qbs = (\n",
    "    dropbacks.group_by(\"id\", maintain_order=True)\n",
    "    .agg(name=pl.col(\"name\").first(), team=pl.col(\"posteam\").last(), n=pl.len(), epa=pl.col(\"qb_epa\").mean())\n",
    "    .filter(pl.col(\"n\") >= 15 * week)\n",
    "    .sort([\"epa\", \"id\"], descending=[True, False])\n",
    "    .head(12)\n",
    "    .reverse()\n",
    ")\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(9, 7))\n",
    "y = list(range(qbs.height))\n",
    "ax.barh(y, qbs[\"epa\"], color=sdvplot.team_colors(qbs[\"team\"].to_list(), \"nfl\"), height=0.7)\n",
    "ax.set_yticks(y, [f\"{name}  \" for name in qbs[\"name\"]])\n",
    "low = min(qbs[\"epa\"].min(), 0)\n",
    "span = qbs[\"epa\"].max() - low\n",
    "ends = qbs[\"epa\"].clip(lower_bound=0)  # where each bar ends on the right\n",
    "for i, (end, value, n) in enumerate(zip(ends, qbs[\"epa\"], qbs[\"n\"], strict=True)):\n",
    "    ax.text(end + 0.1 * span, i, f\"{value:+.2f}  ({n} dropbacks)\", va=\"center\", fontsize=9)\n",
    "ax.set_xlim(low - 0.12 * span, qbs[\"epa\"].max() + 0.45 * span)\n",
    "ax.axvline(0, color=\"#222222\", lw=0.8)\n",
    "sdvplot.add_headshots(ax, [low - 0.06 * span] * qbs.height, y, qbs[\"id\"], league=\"nfl\", id_system=\"gsis\", height=0.075)\n",
    "sdvplot.add_logos(ax, (ends + 0.05 * span).to_list(), y, qbs[\"team\"], league=\"nfl\", season=season, height=0.055)\n",
    "ax.spines[[\"top\", \"right\", \"left\"]].set_visible(False)\n",
    "ax.tick_params(axis=\"y\", length=0)\n",
    "ax.set_xlabel(\"EPA per dropback\")\n",
    "ax.set_title(f\"NFL quarterbacks by EPA per dropback, {season} {through}\", loc=\"left\", fontweight=\"bold\")\n",
    "fig.text(\n",
    "    0.99,\n",
    "    0.01,\n",
    "    f\"Minimum {15 * week} dropbacks. Data: nflverse via sportsdataverse-py\",\n",
    "    ha=\"right\",\n",
    "    fontsize=8,\n",
    "    color=\"grey\",\n",
    ")\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  },
  "sdvplot": {
   "description": "The NFL season to date, rebuilt every week: a power table by EPA per play, offense vs defense and a quarterback leaderboard with headshots.",
   "label": "NFL weekly",
   "position": 1
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}