There are plenty of great answers here already, but what I think would help improve the state of the art would be configurable reward sparsity and configurable priors.
- Reward sparsity means that an easier version of the game gives a score after very few moves, but a more difficult version might take many many moves. It would be useful to see how agents compare against humans as the rewards get sparser.
- Configurable priors are vaguer and are to make it easier to have the game match expectations from prior experience in other games. E.g. researchers could overlay a custom texture on enemies that look like enemies from other games to test and develop better transfer learning algorithms.
- Reward sparsity means that an easier version of the game gives a score after very few moves, but a more difficult version might take many many moves. It would be useful to see how agents compare against humans as the rewards get sparser.
- Configurable priors are vaguer and are to make it easier to have the game match expectations from prior experience in other games. E.g. researchers could overlay a custom texture on enemies that look like enemies from other games to test and develop better transfer learning algorithms.