解析半结构化值

Question

这是我的第一个问题.我试图找到一个答案,但老实说,不能确定我应该使用哪些条款,如果之前已被问到,那就很抱歉.

这就是:我在.txt文件中有数千条记录,格式如下:

(1, 3, 2, 1, 'John (Finances)'),
(2, 7, 2, 1, 'Mary Jane'),
(3, 7, 3, 2, 'Gerald (Janitor), Broflowski'),

... 等等.第一个值是PK,其他3个是外键,第5个是字符串.

我需要在Javascript中将它们解析为JSON(或其他东西),但我遇到了麻烦,因为有些字符串有括号+逗号(在第3条记录上,"Janitor",例如),所以我不能使用子字符串...也许修剪正确的部分,但我想知道是否有一些更聪明的方法来解析它.

任何帮助将非常感激.

谢谢!

Answer 1

你不能(可能不应该读)使用正则表达式.如果括号包含另一对或一个不匹配怎么办？

好消息是您可以轻松地为此构建一个标记器/解析器.我们的想法是跟踪您当前的状态并采取相应的行动.

这是我刚才写的解析器的草图,重点是向您展示一般的想法.如果您对此有任何概念性问题,请与我们联系.

它在这里工作 演示,但我恳求你在理解和修补它之前不要在生产中使用它.

这个怎么运作

那么,我们如何构建解析器:

var State = { // remember which state the parser is at.
     BeforeRecord:0, // at the (
     DuringInts:1, // at one of the integers
     DuringString:2, // reading the name string
     AfterRecord:3 // after the )
};

我们需要跟踪输出和当前工作对象,因为我们将一次解析这些对象.

var records = []; // to contain the results
var state = State.BeforeRecord;

现在,我们迭代字符串,继续进行并读取下一个字符

for(var i = 0;i < input.length; i++){
    if(state === State.BeforeRecord){
        // handle logic when in (
    }
    ...
    if(state === State.AfterRecord){
        // handle that state
    }
}

现在,剩下的就是将它消耗到每个状态的对象中:

• 如果它在(我们开始解析并跳过任何空格
• 阅读所有整数并抛弃 ,
• 在四个整数之后,读取字符串'到下一个'到达它的末尾
• 在字符串之后,读取直到),存储对象,然后再次开始循环.

实施也不是很困难.

解析器

var State = { // keep track of the state
     BeforeRecord:0,
     DuringInts:1,
     DuringString:2,
     AfterRecord:3
};
var records = []; // to contain the results
var state = State.BeforeRecord;
var input = " (1, 3, 2, 1, 'John (Finances)'), (2, 7, 2, 1, 'Mary Jane'), (3, 7, 3, 2, 'Gerald (Janitor), Broflowski')," // sample input

var workingRecord = {}; // what we're reading into.

for(var i = 0;i < input.length; i++){
    var token = input[i]; // read the current input
    if(state === State.BeforeRecord){ // before reading a record
        if(token === ' ') continue; // ignore whitespaces between records
        if(token === '('){ state = State.DuringInts; continue; }
        throw new Error("Expected ( before new record");
    }
    if(state === State.DuringInts){
        if(token === ' ') continue; // ignore whitespace
        for(var j = 0; j < 4; j++){
            if(token === ' ') {token = input[++i]; j--; continue;} // ignore whitespace 
             var curNum = '';
             while(token != ","){
                  if(!/[0-9]/.test(token)) throw new Error("Expected number, got " + token);
                  curNum += token;
                  token = input[++i]; // get the next token
             }
             workingRecord[j] = Number(curNum); // set the data on the record
             token = input[++i]; // remove the comma
        }
        state = State.DuringString;
        continue; // progress the loop
    }
    if(state === State.DuringString){
         if(token === ' ') continue; // skip whitespace
         if(token === "'"){
             var str = "";
             token = input[++i];
             var lenGuard = 1000;
             while(token !== "'"){
                 str+=token;
                 if(lenGuard-- === 0) throw new Error("Error, string length bounded by 1000");
                 token = input[++i];
             }
             workingRecord.str = str;
             token = input[++i]; // remove )
             state = State.AfterRecord;
             continue;
         }
    }
    if(state === State.AfterRecord){
        if(token === ' ') continue; // ignore whitespace
        if(token === ',') { // got the "," between records
            state = State.BeforeRecord;
            records.push(workingRecord);
            workingRecord = {}; // new record;
            continue;
        }
        throw new Error("Invalid token found " + token);
    }
}
console.log(records); // logs [Object, Object, Object]
                      // each object has four numbers and a string, for example
                      // records[0][0] is 1, records[0][1] is 3 and so on,
                      // records[0].str is "John (Finances)"