如何有效地存储大型Java地图？

Question

我很蛮力一场比赛,我需要存储所有位置和结果的数据.数据可能会达到数百Gb.我考虑过SQL,但我担心在紧密循环中查找会破坏性能.如果已知,程序将迭代可能的位置并返回获胜的移动,如果已知所有移动都已丢失则返回最长的丢失序列并检查未知移动的结果.

存储大型商店的最佳方式是什么Map<Long,Long[]> positionIdToBestMoves？我正在考虑SQL或数据序列化.

我想通过强制解决微小的检查器 - 强制Java中的所有可行动作.头寸的上限约为100亿.它们中的大多数都不合理(即比游戏开始时存在的碎片多).大约100亿是一个合理的估计.每个Map<Long, Long[]> position映射Long positionID到Long whiteToMove和Long blackToMove.正值表示位置获胜,应选择导致存储在值中的位置的移动.负值-n意味着位置在大多数n动作中都会丢失.

搜索本身会有这样的递归:

//this is a stub

private Map<Long, Long[]> boardBook =...

//assuming that all winning positions are known
public Long nextMove(Long currentPos, int whiteOrBlack){
Set<Long> validMoves = calculateValidMoves(currentPos, whiteOrBlack);
boolean hasWinner = checkIfValidMoveIsKnownToWin(validMoves, whiteOrBlack);

if(hasWinner){  //there is a winning move - play it
    Long winningMove = getWinningMove(validMoves, whiteOrBlack);
    boardBook.get(currentPos)[whiteOrBlack] = winningMove ;    
    return winningMove ;
    }
boolean areAllPositionsKnown = checkIfAllPositionsKnown(validMoves, whiteOrBlack);
if(areAllPositionsKnown){  //all moves are losing.. choose longest struggle
    Long longestSequenceToDefeat = findPositionToLongestSequenceToDefeat(validMoves, whiteOrBlack);
    int numberOfStepsTodefeat = boardBook.get(longestSequenceToDefeat)[whiteOrBlack];
    boardBook.get(currentPos)[whiteOrBlack] = longestSequenceToDefeat ;
    return longestSequenceToDefeat;
    }

Set<Long> movesToCheck = getUntestedMoves(validMoves, whiteOrBlack);
Long longeststruggle;
int maxNumberOfMovesToDefeat =-1;
for(Long moveTocheck : movesToCheck){
    Long result = nextMove(moveToCheck, whiteOrBlack);
    if(result>0){ //just discovered a winning move
            boardBook.get(currentPos)[whiteOrBlack] = winningMove ;    
            return winningMove ;
        }else {
            int numOfMovesToDefeat = -1*boardBook.get(moveTocheck)[whiteOrBlack];
            if( numOfMovesToDefeat >maxNumberOfMovesToDefeat ){
                 maxNumberOfMovesToDefeat =numOfMovesToDefeat ; 
                 longeststruggle = moveTocheck;
                  }
         }
      }
boardBook.get(currentPos)[whiteOrBlack] = -1*maxNumberOfMovesToDefeat;
return  longeststruggle;
}

Answer 1

您可能想看看纪事报。它是高度优化的键值存储，应该适合您的目的。

或者您可以自己编写存储，但您最终仍然会在幕后执行诸如映射和内存映射文件之类的操作。