Skip to content

Improve multiplayer AI attack targeting with TTK and softmax selection - #11273

Open
Madwand99 wants to merge 12 commits into
Card-Forge:masterfrom
Madwand99:ImproveAIThreatDetection
Open

Improve multiplayer AI attack targeting with TTK and softmax selection#11273
Madwand99 wants to merge 12 commits into
Card-Forge:masterfrom
Madwand99:ImproveAIThreatDetection

Conversation

@Madwand99

Copy link
Copy Markdown
Contributor

This improves multiplayer combat targeting so the AI is less likely to repeatedly focus a single player based on small or incidental score differences.

Changes:

  • Replace the quadratic low-life bonus with a bounded, combat-only time-to-kill (TTK) estimate.
  • Factor both incoming danger and the AI’s ability to finish an opponent into threat evaluation.
  • Use softmax sampling for the primary attack defender, keeping strong threat differences meaningful while varying choices among similarly threatening opponents.
  • Keep ranked fallback defenders so the AI can attack an open opponent when its preferred target is not profitable to attack.
  • Ensure TTK block simulations do not use another player’s hidden information.
  • TTK is intentionally conservative and bounded to ten combat steps. It currently models repeated combat damage only; poison, commander damage, burn, lifegain, and evolving board states remain future extensions.
  • Added a generic softmax function

int lifeDeficit = lowLifeThreshold - life;
score += lifeDeficit * lifeDeficit;
}
score += ComputerUtil.getCombatTtkScore(ComputerUtil.estimateCombatTurnsToKill(ai, opp));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main thoughts for now:
a) this scoring change makes it only look at attack power from AI, where previously being at low life was already enough (assuming other opponents will also consider the same defender)
b) How sure are we that it's really a superior strategy to get rid of the weakest player instead of e.g. slowing the strongest down?
c) I have some doubts looking more than one combat into the future this way won't just lead to other wrong conclusions (attackers/blockers trading or just the board state completely changing)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, some valid concerns there that I also share. It is tricky to get this right. I pushed a change that should help: I’ve kept the existing threat evaluation as the primary signal and changed outgoing TTK into a short-term tactical modifier rather than a general low-life replacement: it only scores kills within one or two combats, so it no longer relies on long static-board projections.
The finishing bonus is also scaled down exponentially when that opponent’s underlying threat is below the table leader’s. This means a low-threat player who is easy to eliminate can help break a close decision, but should not displace a clearly more dangerous opponent that the AI ought to be slowing down. I did not reintroduce a separate low-life bonus, since that would largely double up on the same elimination pressure.

Comment thread forge-ai/src/main/java/forge/ai/ComputerUtil.java
Comment thread forge-ai/src/main/java/forge/ai/ComputerUtil.java
return Integer.MAX_VALUE;
}
final int damage = defender.getLife() - remainingLife;
return damage > 0 ? (defender.getLife() + damage - 1) / damage : Integer.MAX_VALUE;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo we're also losing accuracy here if AI is at very low life so all opponents will basically lead to TTK=1

but it should still matter which opponent causes lower negatives

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think TTK should intentionally collapse all terminal next-combat outcomes to 1: once a player would lose, excess normal damage is not more immediately relevant, and comparing negative life would also mis-rank lethal poison or commander damage that may not reduce life at all. If we need to distinguish between multiple one-turn lethal threats, I think that belongs in the broader board-threat evaluation rather than the TTK value itself.

Comment thread forge-ai/src/main/java/forge/ai/AiAttackController.java Outdated
}
int highestThreat = Collections.max(threatScores.values());

final int temperature = AiProfileUtil.getIntProperty(ai, AiProps.MULTIPLAYER_DEFENDER_SOFTMAX_TEMPERATURE);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't mind a more dynamic approach to determine the score interval that counts as equal threat
but allowing AI to make potentially game losing attacks (even with a rather low probability) just seems wrong

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The small chance for even low-threat opponents to be attacked is intentional. From a recent comment on the Discord: "...Also, humans will take attacks of opportunity. As in, attacking the least threatening guy because they can't get in on the biggest threat. I don't see the AI doing that at all. I will sit there with no creatures out for several turns, just land go. And I wont ever get attacked, they just plow into each other. "

No one should become immune to being attacked just because they sandbag. The small chance of being attacked prevents this. Some randomness added to AI decision making prevents them from becoming unnaturally predictable and makes the game more fun.

@tool4ever

Copy link
Copy Markdown
Contributor

I need to think about if much more complexity for threat scoring is the right approach or if actually supporting AI spreading its damage to multiple players would solve this way more naturally (though maybe also a bit tricky to get right)

@Madwand99

Copy link
Copy Markdown
Contributor Author

I think your recent PR #11331 is complementary rather than a replacement for this PR. It introduces target variety through a dynamic equal-threat window, but it retains the existing threat signal, including the unconditional low-life bonus. This PR improves that signal with combat TTK, limits finishing pressure when a weaker player is serving as a check on a clearer leader, and uses a graded selection probability plus a fallback to an open target when the preferred defender produces no attack.
In particular, master still has a cliff in its decision: opponents below the dynamic score cutoff are never considered, while every eligible opponent is equally likely. That means it can still concentrate all AI attacks on one player whenever that player is sufficiently ahead, and it can still mistake low life for an urgent reason to finish a player even when another opponent is the more immediate danger.
This PR makes the preference proportional rather than binary. An opponent’s ability to kill the AI soon is reflected through combat TTK, while the AI’s ability to finish an opponent is treated as a limited opportunity bonus, not an overriding priority. The finishing bonus is reduced when that low-life player is much less threatening than the leader, preserving them as a possible counterweight rather than having every AI eliminate them.
Softmax then gives near-equal alternatives a meaningful chance without declaring all lower-ranked players impossible targets. Finally, when the preferred target is too well defended to produce a profitable attack, the AI tries its next-ranked target, allowing it to take the available attack opportunity instead of repeatedly declining combat.

In other words: we still need this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants