{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# College football weekly\n",
    "\n",
    "This page is regenerated every week by sdvplot's docs workflow. It rates every FBS team by opponent-adjusted EPA per play, a simple cousin of SP+ built from the\n",
    "play-by-play, for the latest season with data: the season to date during the fall, the final season in the offseason.\n",
    "Data: cfbfastR play-by-play and schedules, read through [sportsdataverse-py](https://py.sportsdataverse.org/)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "Week 0 is in late August, so before then the calendar points at last season. For a season that is not published yet\n",
    "`load_cfb_pbp` warns and returns an empty frame rather than raising, so the helper below turns that into a\n",
    "`NoDataError` and the page steps back one season."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime as dt\n",
    "import warnings\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import polars as pl\n",
    "import sportsdataverse.cfb as cfb\n",
    "from IPython.display import Markdown, display\n",
    "from sportsdataverse.errors import NoDataError\n",
    "\n",
    "import sdvplot\n",
    "\n",
    "today = dt.date.today()\n",
    "current = today.year if today.month >= 8 else today.year - 1\n",
    "COLUMNS = [\n",
    "    \"game_id\",\n",
    "    \"week\",\n",
    "    \"period\",\n",
    "    \"pos_team_id\",\n",
    "    \"def_pos_team_id\",\n",
    "    \"pos_score_diff_start\",\n",
    "    \"scrimmage_play\",\n",
    "    \"EPA\",\n",
    "]\n",
    "\n",
    "\n",
    "def play_by_play(season):\n",
    "    with warnings.catch_warnings():\n",
    "        warnings.simplefilter(\"ignore\")  # \"no data for season(s)\": handled just below\n",
    "        pbp = cfb.load_cfb_pbp([season])\n",
    "    if pbp.is_empty():\n",
    "        raise NoDataError(f\"no {season} play-by-play yet\")\n",
    "    return pbp.select(COLUMNS)\n",
    "\n",
    "\n",
    "try:\n",
    "    season, pbp = current, play_by_play(current)\n",
    "except NoDataError as err:\n",
    "    print(f\"{err}; showing {current - 1} instead\")\n",
    "    season, pbp = current - 1, play_by_play(current - 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3",
   "metadata": {},
   "source": [
    "The schedule gives each team's division, conference and record, and the status line says what the ratings cover."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "schedule = cfb.load_cfb_schedule([season])\n",
    "played = schedule.filter(pl.col(\"completed\"))\n",
    "regular = played.filter(pl.col(\"season_type\") == \"regular\")\n",
    "week = regular[\"week\"].max()\n",
    "left = schedule.filter(~pl.col(\"completed\"))\n",
    "if left.filter(pl.col(\"season_type\") == \"regular\").height:\n",
    "    status = f\"**Season to date:** the {season} season through week {week}.\"\n",
    "    through = f\"through week {week}\"\n",
    "elif left.height:\n",
    "    status = f\"**Postseason:** the {season} regular season is final; bowls and the playoff are under way.\"\n",
    "    through = \"regular season and finished bowls\"\n",
    "else:\n",
    "    status = f\"**Offseason:** the final {season} season, bowls and playoff included.\"\n",
    "    through = \"final\"\n",
    "display(Markdown(status))\n",
    "\n",
    "sides = pl.concat(\n",
    "    [\n",
    "        played.select(\n",
    "            team_id=\"home_id\",\n",
    "            school=\"home_team\",\n",
    "            conference=\"home_conference\",\n",
    "            division=\"home_division\",\n",
    "            win=\"home_winner\",\n",
    "        ),\n",
    "        played.select(\n",
    "            team_id=\"away_id\",\n",
    "            school=\"away_team\",\n",
    "            conference=\"away_conference\",\n",
    "            division=\"away_division\",\n",
    "            win=\"away_winner\",\n",
    "        ),\n",
    "    ]\n",
    ")\n",
    "fbs = (\n",
    "    sides.filter(pl.col(\"division\") == \"fbs\")\n",
    "    .group_by(\"team_id\", maintain_order=True)\n",
    "    .agg(pl.col(\"school\", \"conference\").last(), w=pl.col(\"win\").sum(), l=(~pl.col(\"win\")).sum())\n",
    "    .with_columns(record=pl.format(\"{}-{}\", \"w\", \"l\"))\n",
    ")\n",
    "fbs.sort(\"school\").head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "## 1. Top 25 by adjusted EPA per play\n",
    "\n",
    "The ratings use FBS-against-FBS scrimmage plays, minus garbage time (a lead of more than 43 points in the first\n",
    "quarter, 37 in the second, 27 in the third or 21 in the fourth). One pass of opponent adjustment credits each offensive\n",
    "play for the defense it faced (that defense's EPA allowed per play against the FBS average), and each defensive play\n",
    "for the offense it faced. Net is adjusted offense minus adjusted defense."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "margin = pl.col(\"period\").replace_strict({1: 43, 2: 37, 3: 27, 4: 21}, default=None)  # overtime is never garbage time\n",
    "garbage = pl.col(\"pos_score_diff_start\").abs() > margin\n",
    "ids = fbs[\"team_id\"].implode()\n",
    "plays = pbp.filter(\n",
    "    pl.col(\"scrimmage_play\")\n",
    "    & pl.col(\"EPA\").is_not_null()\n",
    "    & ~garbage.fill_null(False)\n",
    "    & pl.col(\"pos_team_id\").is_in(ids)\n",
    "    & pl.col(\"def_pos_team_id\").is_in(ids)\n",
    ")\n",
    "assert plays.schema[\"pos_team_id\"] == fbs.schema[\"team_id\"]\n",
    "\n",
    "avg = plays[\"EPA\"].mean()\n",
    "offense = plays.group_by(\"pos_team_id\", maintain_order=True).agg(\n",
    "    faced_off=pl.col(\"EPA\").mean()\n",
    ")  # what each defense faced\n",
    "defense = plays.group_by(\"def_pos_team_id\", maintain_order=True).agg(\n",
    "    faced_def=pl.col(\"EPA\").mean()\n",
    ")  # what each offense faced\n",
    "adjusted = (  # keep the play order, so the means below sum in the same order every week\n",
    "    plays.join(defense, on=\"def_pos_team_id\", maintain_order=\"left\").join(\n",
    "        offense, on=\"pos_team_id\", maintain_order=\"left\"\n",
    "    )\n",
    ").with_columns(\n",
    "    adj_off=pl.col(\"EPA\") - (pl.col(\"faced_def\") - avg),\n",
    "    adj_def=pl.col(\"EPA\") - (pl.col(\"faced_off\") - avg),\n",
    ")\n",
    "ratings = (\n",
    "    adjusted.group_by(team_id=\"pos_team_id\", maintain_order=True)\n",
    "    .agg(off=pl.col(\"adj_off\").mean())\n",
    "    .join(\n",
    "        adjusted.group_by(team_id=\"def_pos_team_id\", maintain_order=True).agg(dfn=pl.col(\"adj_def\").mean()),\n",
    "        on=\"team_id\",\n",
    "    )\n",
    "    .with_columns(net=pl.col(\"off\") - pl.col(\"dfn\"))\n",
    "    .join(fbs, on=\"team_id\")\n",
    "    .sort([\"net\", \"team_id\"], descending=[True, False])  # a tiebreaker keeps the weekly re-render stable\n",
    "    .with_row_index(\"rank\", offset=1)\n",
    "    .with_columns(pl.col(\"team_id\").cast(pl.String))  # sdvplot ids are strings; cast the integer, never a float\n",
    ")\n",
    "top25 = ratings.head(25).select(\"rank\", \"team_id\", \"school\", \"conference\", \"record\", \"off\", \"dfn\", \"net\")\n",
    "top25.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7",
   "metadata": {},
   "source": [
    "Conferences get one color each from a qualitative palette, used in the table and the chart below. `gt_sdv_logos`\n",
    "reads the ESPN team ids straight from the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from great_tables import GT\n",
    "from matplotlib.colors import to_hex\n",
    "\n",
    "from sdvplot.great_tables import gt_save_crop, gt_sdv_logos, gt_theme_ncaa\n",
    "\n",
    "conferences = sorted(fbs[\"conference\"].unique().drop_nulls())\n",
    "qualitative = [c for i, c in enumerate(plt.get_cmap(\"tab10\").colors) if i != 7] + list(plt.get_cmap(\"Dark2\").colors[3:])\n",
    "CONF_COLORS = {conf: to_hex(color) for conf, color in zip(conferences, qualitative, strict=False)}  # tab10 minus grey\n",
    "\n",
    "gt = (\n",
    "    GT(top25, id=\"cfb-top25\")  # a fixed id: great_tables otherwise draws a random one each run\n",
    "    .tab_header(f\"College football top 25, {season}\", f\"Opponent-adjusted EPA per play, {through}\")\n",
    "    .fmt_number([\"off\", \"dfn\", \"net\"], decimals=3, force_sign=True)\n",
    "    .data_color(\"conference\", palette=[CONF_COLORS[c] for c in conferences], domain=conferences)\n",
    "    .data_color(\"net\", palette=[\"#f7f7f7\", \"#2e8b57\"], domain=[0, top25[\"net\"].max()])\n",
    "    .tab_spanner(\"Adjusted EPA per play\", [\"off\", \"dfn\", \"net\"])\n",
    "    .cols_label(\n",
    "        rank=\"\",\n",
    "        team_id=\"\",\n",
    "        school=\"Team\",\n",
    "        conference=\"Conference\",\n",
    "        record=\"Record\",\n",
    "        off=\"Offense\",\n",
    "        dfn=\"Defense\",\n",
    "        net=\"Net\",\n",
    "    )\n",
    "    .tab_source_note(\n",
    "        \"Data: cfbfastR via sportsdataverse-py. FBS vs FBS scrimmage plays, garbage time removed; \"\n",
    "        \"defense is EPA allowed (lower is better).\"\n",
    "    )\n",
    ")\n",
    "gt = gt_theme_ncaa(gt_sdv_logos(gt, \"team_id\", league=\"cfb\", height=26))\n",
    "gt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9",
   "metadata": {},
   "source": [
    "`gt_save_crop` renders the same table to a trimmed PNG, ready to post."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {
    "sdvplot_gallery": {
     "alt": "College football top 25 table with team logos, conference colors, record and opponent-adjusted EPA per play.",
     "title": "College football top 25 by adjusted EPA"
    },
    "tags": [
     "gallery"
    ]
   },
   "outputs": [],
   "source": [
    "gt_save_crop(gt, width=900)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11",
   "metadata": {},
   "source": [
    "## 2. Every FBS team, by conference\n",
    "\n",
    "One row per conference, ordered by the conference's average rating, each team's logo at its net rating (alternately\n",
    "nudged up and down so neighbors overlap less). The line in the conference color spans the conference from its lowest\n",
    "to its highest team."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {
    "sdvplot_gallery": {
     "alt": "Every FBS team logo placed by its opponent-adjusted net EPA per play, one row per conference.",
     "title": "FBS teams by conference"
    },
    "tags": [
     "gallery"
    ]
   },
   "outputs": [],
   "source": [
    "by_conf = ratings.filter(pl.col(\"conference\").is_not_null())\n",
    "order = (\n",
    "    by_conf.group_by(\"conference\", maintain_order=True)\n",
    "    .agg(pl.col(\"net\").mean())\n",
    "    .sort(\"net\", \"conference\")[\"conference\"]\n",
    "    .to_list()\n",
    ")\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 7.5))\n",
    "for row, conf in enumerate(order):\n",
    "    teams = by_conf.filter(pl.col(\"conference\") == conf).sort(\"net\", \"team_id\")\n",
    "    ax.hlines(row, teams[\"net\"].min(), teams[\"net\"].max(), color=CONF_COLORS[conf], lw=7, alpha=0.35, zorder=1)\n",
    "    ax.plot(teams[\"net\"].mean(), row, marker=\"|\", markersize=26, mew=2.5, color=CONF_COLORS[conf], zorder=2)\n",
    "    rows = [row + (0.17 if i % 2 else -0.17) for i in range(teams.height)]  # alternate up and down: less overlap\n",
    "    sdvplot.add_logos(ax, teams[\"net\"], rows, teams[\"team_id\"], league=\"cfb\", season=season, height=0.042)\n",
    "ax.set_yticks(range(len(order)), order)\n",
    "ax.set_ylim(-0.7, len(order) - 0.3)\n",
    "ax.margins(x=0.04)\n",
    "ax.axvline(0, color=\"grey\", lw=0.8, ls=\"--\")\n",
    "ax.spines[[\"top\", \"right\", \"left\"]].set_visible(False)\n",
    "ax.tick_params(axis=\"y\", length=0)\n",
    "ax.set_xlabel(\"Net adjusted EPA per play (the tick marks the conference average)\")\n",
    "ax.set_title(f\"FBS teams by conference, {season} {through}\", loc=\"left\", fontweight=\"bold\")\n",
    "fig.text(0.99, 0.01, \"Data: cfbfastR via sportsdataverse-py\", ha=\"right\", fontsize=8, color=\"grey\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13",
   "metadata": {},
   "source": [
    "## 3. Offense and defense of the top 25\n",
    "\n",
    "The two halves of the rating for the top 25, drawn with plotnine. `geom_sdv_logos` takes the ESPN ids through the\n",
    "`team` aesthetic; the defense axis is reversed so the better defenses sit higher."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {},
   "outputs": [],
   "source": [
    "from plotnine import aes, element_text, ggplot, labs, scale_x_continuous, scale_y_reverse, theme, theme_minimal\n",
    "\n",
    "from sdvplot.plotnine import geom_mean_lines, geom_sdv_logos\n",
    "\n",
    "(\n",
    "    ggplot(top25.to_pandas(), aes(\"off\", \"dfn\", x0=\"off\", y0=\"dfn\", team=\"team_id\"))\n",
    "    + geom_mean_lines(color=\"grey\")\n",
    "    + geom_sdv_logos(league=\"cfb\", season=season, height=0.07)\n",
    "    + scale_x_continuous(expand=(0.06, 0))  # logos do not widen the limits: leave room for the outermost ones\n",
    "    + scale_y_reverse(expand=(0.08, 0))\n",
    "    + labs(\n",
    "        x=\"Adjusted offense: EPA per play\",\n",
    "        y=\"Adjusted defense: EPA allowed per play (reversed)\",\n",
    "        title=f\"How the top 25 get there, {season} {through}\",\n",
    "        caption=\"Dashed lines: the top-25 averages. Data: cfbfastR via sportsdataverse-py\",\n",
    "    )\n",
    "    + theme_minimal()\n",
    "    + theme(figure_size=(8, 6), plot_title=element_text(weight=\"bold\"))\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  },
  "sdvplot": {
   "description": "The college football season to date, rebuilt every week: a top 25 by opponent-adjusted EPA per play with logos and conference colors, every FBS team by conference, and the top 25's offense and defense.",
   "label": "College football weekly",
   "position": 2
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}