I have been trying a different representation of the board with some success. I use [my_strength, enemy_strength, open_strength, production], where my_strength is strength * owner where owner is myID.
If you have two separate fields for owner and strength, the model is expected to learn the relationship between the strength and the owner to determine the appropriate move. I feel making it explicit in your training set is more efficient and easier to debug that you have the correct input and you're doing the correct transformations. For each position, I create a relative frame where the square for which the model is deciding is in the center
I also trained two separate models, one to determine if a move should take place, and another to determine what the move should be. That seems to work much better.
Since the number of training examples can be very large, creating a generator really helped me manage my training. For the validation set, I took the first N inputs off to the side and had the generator skip them so ensure the model never sees them. The N inputs span a number of games so there isn't bias in terms of validating against just the beginning moves of a game.
I've posted the gist here. For some reason though, it doesn't do as well as when I had saved down the games so I'm still trying to see if there are any bugs. The saved down training set that worked better was able to shuffle the training data, but I feel that my samples per epoch is large enough to offset that.
I was able to create a bot that uses a model to determine if a move should occur and another to determine what the move should be. It has some weird behavior but performs very well in the beginning. But it begins acting very strange towards the end as the bot is commanding more squares. I think its due to the fact that the training set is essentially the same (own all the squares to the left and right of the square being evaluated). Then it is unable to make the move. Still not sure how to handle that but I was thinking about either expanding the visible distance, pooling or having a special case for those circumstances where the square just heads to the closest exit.