如何使用辅助功能服务"为用户采取行动"？

Question

背景

遥想几年前,我问的TeamViewer如何允许用户控制设备,而无需与设备正常交互.我被告知这是一个特殊的"后门",制造商专门为这个应用程序提供,并且只能使用root权限用于其他应用程序.

看到像"飞行模式快捷方式"这样的应用程序允许切换飞行模式,通过自动导航到其屏幕并切换开关,它让我意识到这种情况已经改变.

问题

在文档中说:

从Android 4.0(API级别14)开始,辅助功能服务可以代表用户执行操作,包括更改输入焦点和选择(激活)用户界面元素.在Android 4.1(API级别16)中,操作范围已扩展为包括滚动列表和与文本字段交互.辅助功能服务还可以执行全局操作,例如导航到主屏幕,按"返回"按钮,打开通知屏幕和最近的应用程序列表.Android 4.1还包括一种新型焦点Accessibilty Focus,它使所有可见元素都可由辅助功能服务选择.

这些新功能使辅助功能服务的开发人员可以创建替代导航模式,如手势导航,并为残障用户提供对其Android设备的更好控制.

但是没有关于如何使用它的更多信息.只有我发现的样本位于底部,但这些样本非常陈旧,是apiDemos包的一部分.

这个问题

如何创建可以查询,聚焦,单击,输入文本以及执行其他UI相关操作的服务？

Answer 1

通过实施AccessibilityService(https://developer.android.com/training/accessibility/service.html),您可以访问这些功能.

您可以检查或对用户最后交互的元素执行操作,也可以检查当前活动的整个应用程序.

通过实现拦截用户事件onAccessibilityEvent(AccessibilityEvent event),在这里您可以检索虚拟视图(表示原始视图),event.getSource()然后使用getClassName()或getText()在文档中找到的任何内容进行检查.

通过调用getRootInActiveWindow()并遍历virtaul视图树来检查整个应用程序getRootInActiveWindow().getChild(index).

both getRootInActiveWindow()和event.getSource()return AccessibilityNodeInfo,你可以在其上调用performAction(action)并执行类似Click,Set Text等操作.

示例:Play商店

一旦您打开了Play商店应用,搜索"facebook"应用并在Play商店中打开它的页面.

    @Override
    public void onAccessibilityEvent(final AccessibilityEvent event) {

        AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
        //Inspect app elements if ready
        if (rootInActiveWindow != null) {
            //Search bar is covered with textview which need to be clicked
            List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
            if (searchBarIdle.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
                searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
            }
            //Check is search bar is visible
            List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
            if (searchBars.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBars.get(0);
                //Check is searchbar have the required text, if not set the text
                if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
                    Bundle args = new Bundle();
                    args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
                    searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
                } else {
                    //There is no way to press Enter to perform search, so find corresponding suggestion and click
                    List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
                    for (AccessibilityNodeInfo suggestion : searchSuggestions) {
                        if(suggestion.getText().toString().equals("Facebook")) {
                            //We found textview, but its not clickable, so we should perform the click on the parent
                            AccessibilityNodeInfo clickableParent = suggestion.getParent();
                            clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                        }
                    }
                }
            }


        }
   }

编辑:完整代码如下:

MyAccessibilityService

public class MyAccessibilityService extends AccessibilityService {

    @Override
    public void onCreate() {
        super.onCreate();
        Log.d("MyAccessibilityService", "onCreate");
    }

    @Override
    public void onAccessibilityEvent(final AccessibilityEvent event) {
        Log.d("MyAccessibilityService", "onAccessibilityEvent");
        AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
        //Inspect app elements if ready
        if (rootInActiveWindow != null) {
            //Search bar is covered with textview which need to be clicked
            List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
            if (searchBarIdle.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
                searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
            }
            //Check is search bar is visible
            List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
            if (searchBars.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBars.get(0);
                //Check is searchbar have the required text, if not set the text
                if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
                    Bundle args = new Bundle();
                    args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
                    searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
                } else {
                    //There is no way to press Enter to perform search, so find corresponding suggestion and click
                    List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
                    for (AccessibilityNodeInfo suggestion : searchSuggestions) {
                        if (suggestion.getText().toString().equals("Facebook")) {
                            //We found textview, but its not clickable, so we should perform the click on the parent
                            AccessibilityNodeInfo clickableParent = suggestion.getParent();
                            clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                        }
                    }
                }
            }
        }
    }

    @Override
    public void onInterrupt() {
    }
}

AndroidManifest.xml中

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
          package="com.example.findfacebookapp">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN"/>

                <category android:name="android.intent.category.LAUNCHER"/>
            </intent-filter>
        </activity>
        <service
            android:name=".MyAccessibilityService"
            android:label="@string/accessibility_service_label"
            android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
            <intent-filter>
                <action android:name="android.accessibilityservice.AccessibilityService"/>
            </intent-filter>

            <meta-data
                android:name="android.accessibilityservice"
                android:resource="@xml/accessibility_service_config"/>
        </service>
    </application>

</manifest>

RES/XML/accessibility_service_config.xml

<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes="typeAllMask"
    android:accessibilityFeedbackType="feedbackAllMask"
    android:accessibilityFlags="flagDefault"
    android:canRequestEnhancedWebAccessibility="true"
    android:canRetrieveWindowContent="true"
    android:description="@string/app_name"
    android:notificationTimeout="100"/>

主要活动

public class MainActivity extends AppCompatActivity {

    public void onEnableAccClick(View view) {
        startActivityForResult(new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS), 1);
    }

}