使用AccessibilityService从(Facebook)RecyclerView中读取文本

Question

如何查看非聚焦的RecyclerView子项(项目)的文本？

我的辅助功能服务应该从屏幕上读出包含关键字的文本.这在大多数情况下效果很好,除非我处理RecyclerViews,特别是那些在Facebook应用程序中的.

Facebook提要和评论屏幕都基于RecyclerView,似乎都不能让我从他们的辅助功能事件中访问我需要的文本.

我尝试过的:

public void onAccessibilityEvent(AccessibilityEvent event) {

    //I've tried looping through event.getText() and it does not contain the text entries

    //I've also tried the following (with various null checks)
    AccessibilityRecordCompat record = AccessibilityEventCompat.asRecord(event);
    AccessibilityNodeInfoCompat source = record.getSource();
    List<AccessibilityNodeInfoCompat> nodes = source.findAccessibilityNodeInfosByText("...");
    //(and loop through the entries)

    //I've also tried limiting to the class name "android.support.v7.widget.RecyclerView"
    //and looking at parcelableData

    //In the facebook app, the following is always null too
    source.findFocus(AccessibilityNodeInfoCompat.FOCUS_ACCESSIBILITY);

}

似乎没有地方包含我需要的数据.作为从帖子打开评论屏幕/叠加层的示例,我可以看到所有文本内容,除了评论本身(在回收者视图中)

我可以看到的事件详细信息的日志:

以下是我在帖子中看到的评论叠加在一个帖子上的评论

Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.support.v7.widget.RecyclerView; Text: []; ContentDescription: null; ItemCount: 100; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: true; BeforeText: null; FromIndex: 13; ToIndex: 24; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_STATE_CHANGED;   PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: X.BGT; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: true; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.support.v7.widget.RecyclerView; Text: []; ContentDescription: null; ItemCount: 1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: 0; ToIndex: 0; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.view.ViewGroup; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: Photo; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.ImageButton; Text: []; ContentDescription: Stickers; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: false; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_CONTENT_CHANGED; PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: android.widget.TextView; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
Event: EventType: TYPE_WINDOW_STATE_CHANGED;   PackageName: com.facebook.katana; MovementGranularity: 0; Action: 0 [ ClassName: X.BGT; Text: [Be the first to like this., Like Button, Photo Comment, Write a comment…, Open GIF search, Stickers]; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: true; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0

我也尝试过探索源代码的子代,但只显示了ViewGroups(没有视图),并且总是使用text null

Ancestor for 0 = android.view.accessibility.AccessibilityNodeInfo@800a418a; boundsInParent: Rect(0, 0 - 1056, 236); boundsInScreen: Rect(12, 244 - 1068, 480); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]
Ancestor for 0, 0 = android.view.accessibility.AccessibilityNodeInfo@800a3647; boundsInParent: Rect(0, 0 - 1056, 236); boundsInScreen: Rect(12, 244 - 1068, 480); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: true; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_LONG_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]
Ancestor for 0, 0, 0 = android.view.accessibility.AccessibilityNodeInfo@800a3a08; boundsInParent: Rect(0, 0 - 828, 58); boundsInScreen: Rect(204, 262 - 1032, 320); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: true; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_LONG_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]
Ancestor for 0, 0, 0, 0 = android.view.accessibility.AccessibilityNodeInfo@800a454b; boundsInParent: Rect(0, 0 - 268, 58); boundsInScreen: Rect(204, 262 - 472, 320); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]
Ancestor for 0, 0, 1 = android.view.accessibility.AccessibilityNodeInfo@800a3dc9; boundsInParent: Rect(0, 0 - 828, 48); boundsInScreen: Rect(204, 320 - 1032, 368); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: true; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_LONG_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]
Ancestor for 0, 0, 2 = android.view.accessibility.AccessibilityNodeInfo@800a490c; boundsInParent: Rect(0, 0 - 828, 80); boundsInScreen: Rect(204, 382 - 1032, 462); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]
Ancestor for 0, 0, 2, 0 = android.view.accessibility.AccessibilityNodeInfo@800a4ccd; boundsInParent: Rect(0, 0 - 93, 44); boundsInScreen: Rect(507, 394 - 600, 438); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]
Ancestor for 0, 0, 2, 1 = android.view.accessibility.AccessibilityNodeInfo@800a508e; boundsInParent: Rect(0, 0 - 114, 44); boundsInScreen: Rect(632, 394 - 746, 438); packageName: com.facebook.katana; className: android.view.ViewGroup; text: null; error: null; maxTextLength: -1; contentDescription: null; viewIdResName: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null]

我不确定这是否可能,或者我可能采取了完全错误的方法.我注意到在TalkBack模式下,用户必须在阅读之前单击/聚焦RecyclerView子项的各个元素,这不是我的服务选项(应该找到感兴趣的项目)

Answer 1

我遇到过嵌套滚动视图的问题，通过递归查找所有视图来解决，我想，您已经尝试过这个。这是适合我的情况的解决方案，您可以尝试它也可能适合您。

private AccessibilityNodeInfo accessbiltyContent(AccessibilityNodeInfo parentView,String strAddress) throws Exception {

    int childCount = parentView.getChildCount();


    for (int i = 0; i < childCount; ++i) {
        AccessibilityNodeInfo child = parentView.getChild(i);
        // child=null;
        if (child != null) {

            String strToAddress=strAddress+" > "+child.getClassName().toString();
            String nodeText="";

            if(child.getText() != null)
                nodeText=child.getText().toString();


            Log.v(TAG,"values =" +child.getClassName().toString(),nodeText,strToAddress));
            accessbiltyContent(child,strToAddress);

        }
        else {
            return null;
        }
    }

    return null;


}

它将遍历父级中的所有视图并打印它们以查找。