Android SQLite和庞大的数据集

Question

我们正在为SQLite数据库中具有数百兆字节HTML的客户端创建一个应用程序.我们已经实现了一种查询此数据的方法,并以合理快速的方式滚动浏览所有数据.问题是某些数据库有非常大的查询(20,000多行),当我们在用户滚动时增加查询时,我们会看到错误.所以我想问题是,我们在Android中查询和显示数万行数据时有哪些选择？

这是我们看到的堆栈跟踪:

09-10 19:19:12.575: WARN/IInputConnectionWrapper(640): showStatusIcon on inactive InputConnection
09-10 19:19:18.226: DEBUG/dalvikvm(640): GC freed 446 objects / 16784 bytes in 330ms
09-10 19:19:32.886: ERROR/CursorWindow(19416): need to grow: mSize = 1048576, size = 36, freeSpace() = 30, numRows = 17717
09-10 19:19:32.896: ERROR/CursorWindow(19416): not growing since there are already 17717 row(s), max size 1048576
09-10 19:19:32.916: ERROR/CursorWindow(19416): The row failed, so back out the new row accounting from allocRowSlot 17716
09-10 19:19:33.005: ERROR/Cursor(19416): Failed allocating fieldDir at startPos 0 row 17716
09-10 19:19:35.596: DEBUG/Cursor(19416): finish_program_and_get_row_count row 24315
09-10 19:19:41.545: DEBUG/dalvikvm(698): GC freed 2288 objects / 126080 bytes in 260ms
09-10 19:19:43.705: WARN/KeyCharacterMap(19416): No keyboard for id 0
09-10 19:19:43.717: WARN/KeyCharacterMap(19416): Using default keymap: /system/usr/keychars/qwerty.kcm.bin
09-10 19:20:04.705: ERROR/CursorWindow(19416): need to grow: mSize = 1048576, size = 17, freeSpace() = 3, numRows = 17094
09-10 19:20:04.716: ERROR/CursorWindow(19416): not growing since there are already 17094 row(s), max size 1048576
09-10 19:20:04.726: ERROR/Cursor(19416): Failed allocating 17 bytes for text/blob at 17093,2
09-10 19:20:05.656: DEBUG/Cursor(19416): finish_program_and_get_row_count row 5257
09-10 19:24:54.685: DEBUG/dalvikvm(637): GC freed 9297 objects / 524176 bytes in 247ms
09-10 19:32:07.656: DEBUG/dalvikvm(19416): GC freed 9035 objects / 495840 bytes in 199ms

这是我们的CursorAdapter代码:

    private class MyAdapter extends ResourceCursorAdapter {

    public MyAdapter(Context context, Cursor cursor) {
        super(context, R.layout.my_row, cursor);        
    }

    public void bindView(View view, Context context, Cursor cursor) {                
        RowData data = new RowData();
        data.setName(cursor.getInt(cursor.getColumnIndex("name")));

        TextView tvItemText = (TextView)view.findViewById(R.id.tvItemText);
        tvItemText.setText(data.getName());

        view.setTag(data);
    }

    @Override
    public Cursor runQueryOnBackgroundThread(CharSequence constraint) {
        /* Display the progress indicator */
        updateHandler.post(onFilterStart);

        /* Run the actual query */               
        if (constraint == null) {
            return myDbObject.getData(null);                     
        }

        return myDbObject.getData(constraint.toString());                
    }            
}

Answer 1

我们在Android中查询和显示数万行数据时有哪些选择？

你的意思是除了告诉你在3.5英寸液晶显示器上读取20,000多行是蝙蝠 - 鸟粪疯狂吗？;-)

它看起来像是CursorWindow在幕后的某个地方使用,有问题管理> 17,000行.这可能是两件事之一:

1. 你没有堆空间.使用16MB非压缩堆,以及a Cursor将整个结果集保存在堆中的事实,这不是不可能的.
2. CursorWindow 仅支持1MB的数据,这是错误消息更直接的建议.

如果有一种逻辑方法可以将查询划分为离散块,则可以进行增量查询并使用CursorJoiner它们将它们拼接在一起,看看是否有帮助.

但是,严肃地说,3.5英寸屏幕上的20,000多行,在最接近马力的12年PC上的设备上,确实需要很多.

Answer 2

如果你真的需要,你也可以拆分你的数据并读取这样的块:

   int limit = 0;
   while (limit + 100 < numberOfRows) {
       //Compose the statement
       String statement = "SELECT * FROM Table ORDER someField LIMIT '"+ limit+"', 100";
       //Execute the query
       Cursor cursor = myDataBase.rawQuery(statement, null);
       while (cursor.moveToNext()) {
           Product product = new Product();
           product.setAllValuesFromCursor(cursor);
           productsArrayList.add(product);
      }
      cursor.close();
      limit += 100;
 }

 //Compose the statement
 String statement = "SELECT * FROM Table ORDER someField LIMIT '"+  (numberOfRows - limit)+"', 100";
 //Execute the query
 Cursor cursor = myDataBase.rawQuery(statement, null);

 while (cursor.moveToNext()) {
     Product product = new Product();
     product.setAllValuesFromCursor(cursor);
     productsArrayList.add(product);
 }
 cursor.close();

如果您有索引表,它在2秒内工作5k行.

谢谢,阿克德