Completed

NBA 2K26 Gameplay Automation

Computer vision and a state machine that play repetitive MyPLAYER shooting challenges on their own.

Overview

An automated gameplay system for NBA 2K26. The PS5 is streamed over Remote Play to an iPhone and mirrored to a Mac, where each frame is analysed to decide what to do next. A YOLO model detects players, colour-based vision picks out court features like the half-court line, and a state machine coordinates the actions.

Highlights

Closed loop: observe, decide, act, verify, using the on-screen score and court state to confirm each action worked
State machine handling positioning, step-backs, shooting, defending and rebounds
YOLO object detection trained to identify players, combined with colour-based court feature detection

Built with

Python
YOLO
Computer Vision
Image Processing
State Machines
Optuna
scikit-learn
XGBoost
PyTorch

Machine learning experiments

After building the automation, I investigated whether the system could learn to shoot more effectively, then put the optimal shooting parameters it found into the automation in place of hand-picked ones.

Shooting animation optimisation

Completed

Automated experiments across the three parameters that control the shooting motion: downward drag speed, time held at the bottom, and upward drag speed. Optuna searched the combinations sequentially, using shot success as the objective, to find and validate the most effective shooting technique.

Python
pandas
NumPy
Optuna

Shooting location optimisation

Completed

With the shooting technique fixed, this experiment looked at which locations and movement strategies give the highest success rate. Each attempt recorded the player's X/Y distance from the rim, whether a step-back was used and whether the shot went in. I modelled shot success with scikit-learn and XGBoost, plus a PyTorch neural network for comparison. The model identified the best shooting location and showed that step-backs lowered success, so I moved the automation to the optimum position and dropped the step-back.

scikit-learn
XGBoost
PyTorch
CSV