The 2024 KAIST-POSTECH Science War (KPSW), AI competition featured AI agents playing “Super Smash Bros. Melee” on the Nintendo GameCube. I want to share the challenges we faced during preparation as the lead of the POSTECH AI team.

POSTECH AI team spent four months for discussing with the KAIST AI team the game to choose for our AI agents. After the Spring semester began, we concluded that “Super Smash Bros. Melee” was the best choice. We then formed a joint programming team to ensure we could create AI agents capable of playing the game well.

During the joint development period for “Super Smash Bros. Melee,” we created an environment where the AI agent could interact with the game. We conducted experiments in a controlled setting to verify the agent’s learning capabilities, testing various training methods and network architectures. Most of our time was spent running these experiments and continuously improving the AI agents. By June, we concluded that even without further joint development, each team could continue developing capable AI agents independently. At that point, we decided to end the collaboration.

After the joint development period, the POSTECH AI team was organized into five sub-teams: four focused on experimenting with training algorithms, and one performing model evaluation. We held weekly meetings to share experiment results and provide feedback to each other.

Programming an AI agent to play a fighting game is a highly challenging task. The AI model needs to quickly determine optimal actions within a short time frame and adapt to various opponents and strategies. Additionally, it relies on a reinforcement learning approach, which involves gaining experience through direct gameplay. As a result, this process demands significant training time and computing resources.

Given the game’s limitless strategies, we even used Reinforcement Learning from Human Feedback (RLHF) to address strategic weaknesses. Both AI and human players directly played the game to refine the AI’s strategy. Human players dug into AI’s weakness, giving insight into AI to reinforce the weakness of its strategy. One team member even spent 24 hours playing the game to train the agent continuously. 

Our training server was unstable, often causing interruptions during the training process. Starting in June, we frequently encountered GPU errors, which caused significant frustration. Our “Link” agent was expected to become our ace. However, due to a 10-hour server downtime, the “Link” agent experienced some performance loss, allowing our “Luigi” agent to take the lead as our ace. Nevertheless, the “Luigi” agent performed well during the competition.

Furthermore, just 10 minutes before the final AI agent submission, the server responsible for evaluating the model encountered an error, causing us to lose the crucial final evaluation results, and we had to submit the model based on previous evaluations.

During development, we had to change the game for the AI competition, conducted countless experiments, training, and evaluation processes, and navigated through countless errors, experiencing a rough development journey. Despite these obstacles, we are grateful that the results turned out well, and we believe that every team member gained valuable experience throughout the preparation for the 2024 KAIST-POSTECH competition.

This concludes the behind-the-scenes story of our journey preparing for the KAIST-POSTECH AI competition. The best of the best, the POSTECH AI team, will be back with more exciting AI matches in the future.