Wait for it… IdlingResource and

【转载自】Wait for it… IdlingResource and ConditionWatcher

In this post we would like to raise a topic of Espresso synchronisation with asynchronous operations occurring in Android application. We will try to track data flow responsible for synchronisation and present it in the simplified and complex way. Furthermore we wanted to present our tool - ConditionWatcher and compare it to currently existing within Espresso framework IdlingResource.

---

Espresso and synchronisation

According to Espresso cheat sheet most of Espresso operations starts with one of two matchers: ObjectMatcher or ViewMatcher which are used to localise element of a screen. On element found in such way test author is able to invoke one of two methods: perform() and check(). Let’s take a closer look at ViewInteraction class where those classes are located, to understand how Espresso works.

• doPerform() called by method perform(final ViewAction… viewActions) on each ViewAction sent as parameter
• body of method check()

Both blocks of code use runSynchronouslyOnUiThread(Runnable runnable) method. It adds ViewAssertion or ViewAction to global queue by transforming Runnable to FutureTask and addingitExecutor working on Android main thread.

When FutureTask is being executed, it calls method of UiController called loopMainThreadUntilIdle() before it reaches code related to previously sent ViewAssertion or ViewAction. UiController is responsible for MotionEvents (used to perform UI operations e.g click, scroll, swipe, drag and more). Method loopMainThreadUntilIdle() postpones execution of task’s body until app state is considered idle.

But if we look deeper into class implementing UiController interface — which is UiControllerImpl,we can find more information about how Espresso prolongs app’s transition to idle state. There is a set of enums called IdleCondition:

• DELAY_HAS_PAST:used in UiController’s method loopMainThreadForAtLeast(long millisDelay) which usage unfortunately can’t be tracked
• ASYNC_TASK_HAVE_IDLED: represent wait for AsyncTasks of AndroidSDK to finish. Class responsible for AsyncTask monitoring is called AsyncTaskPoolMonitor
• COMPAT_TASKS_HAVE_IDLED: represents wait for AsyncTasks of AppCompat lib. Class responsible for monitoring of those AsyncTasks is also AsyncTaskPoolMonitor but it’s different instance than one used for ASYNC_TASK_HAVE_IDLED IdleConditions,
• KEY_INJECT_HAS_COMPLETED: represents wait for any kind of screen interaction which makes device react to KeyPress
• MOTION_INJECTION_HAS_COMPLETED: represents wait for any kind of screen interaction which makes device react to MotionEvent
• DYNAMIC_TASKS_HAVE_IDLED: represents wait for any kind of resources registered by user in IdlingResourceRegistry. More about this kind of IdleCondition type in next paragraph.

To sum up:

Test author is able to perform actions and asserts on elements of the screen. Those actions are queued with all other operations performed on app’s main thread. Before action/assert code is being executed, it will wait until app state is considered to be idle by Espresso framework. App in order to reach idle state needs to meet all idling conditions. Idling conditions require all AsyncTasks, KeyEvents, MotionEvents to be finished and test author stated conditions to be met.

Time to ask a question. Not everyone are using AsyncTasks nowadays, there are many other ways to perform action asynchronously. How to synchronise such “man-made” operations with Espresso?

---

IdlingResource

IdlingResource is an object created by test author which contains logical condition that must be additionally met in order for app to achieve idle state. Such created object needs to be registered in IdlingResourceRegistryto be noticed by Espresso. From Espresso’s perspective, IdlingResource is a representation of dynamic resource of Android application under test. It can be any operation performed by the app e.g REST requests, database operations, animations, data loading into list and more. UiController handles wait for conditions stated inside registered IdlingResources with usage of DYNAMIC_TASKS_HAVE_IDLED IdlingCondition enum.

How does it work?

1. Setup Espresso according to documentation. IdlingResource is not included in core lib, it’s an extension.
2. Class which implements IdlingResource interface must be created.
3. IdlingResource interface contains three abstract method which implementation needs to be provided in order for IdlingResource to work properly. Those methods are: getName(), isIdleNow(), and registerIdleTransitionCallback(ResourceCallback callback).
4. Method getName() cannot be null and can return any String. It is used by IdlingResourceRegistry to prevent two resources with the same name being registered at the same time.
5. Create local ResourceCallback field inside IdlingResource and override registerIdleTransitionCallback(ResourceCallback callback). This method is used by IdlingResourceRegistry during IdlingResource registration. IdlingResourceRegistry will send instance of ResourceCallback to IdlingResource created and registered by test author what will connect it to whole app idling process. Such received ResourceCallback should be saved in previously created local field.
6. Method isIdleNow() is core and most important part of IdlingResource. It gives IdlingResourceRegistry information about current status of condition stated within IdlingResource. It should contain logical expression that changes value from false to true, after expected by test author shift had occurred. Furthermore it is IMPORTANT to call there onTransitionToIdle() method ResourceCallback after logical expression starts returning true to notify that change within dynamic resource has occurred.
7. Created IdlingResource needs to be added to IdlingResourceRegistry via registerResources() method (static method of Espresso class). After that it will be taken into consideration during next usage of perform() or check() methods where loopMainThreadUntilIdle() occurs. Method isIdleNow() will be called with fixed interval until transition to idle will be reported by all registered resources or timeout will be thrown.
8. After registered IdlingResources are used up and app reaches idle status, remember that they are still inside IdlingResourceRegistry’s list which is a Singleton shared between all test cases. Consequently it is needed to remember to call unregisterResources() (static method of Espresso class) when they are not needed anymore in order to avoid malfunctions.

Example of fully created IdlingResource which waits until custom-made loading Dialog with ProgressBar disappear from view hierarchy:

Example of more IdlingResources and their usages inside automation tests can be found in our sample project of ConditionWatcher.

---

Possible issues and inconveniences

During our work with Espresso we came across things that could be found only by inspecting code of Espresso classes and that we couldn’t fully understand only by using available documentation. It caused us a few issues we would like to share.

IdlingResource scanning interval

When IdlingResource is registered in IdlingResourceRegistry, it will have it’s isIdleNow() method called in a fixed interval to check if custom made idling condition has been met yet. When this process takes too long IdlingResourceTimeoutException will be thrown. Information about Espresso timeouts is stored inside IdlingPolicies class.

As can be observed default timeout for IdlingResource is 26 seconds. This value can be changed by calling setIdlingResourceTimeout(long timeout, TimeUnit unit). But there is more useful information. The second parameter named dynamicIdlingResourceErrorPolicy stores information about isIdleNow() method call interval. This interval equals 5 seconds and can’t be modified.

Why is that a problem? Imagine there is a test set of 500 tests. Application’s main screen has menu with cool animation that takes 600 milliseconds to reveal buttons. If IdlingResource is used to wait before buttons are visible, test author would waste around 4,4 seconds because first condition check of isIdleNow() will return false and the next one will be after 5 seconds. Now imagine losing 4,4 seconds in most of 500 tests (~37min in assumption there is only one IdlingResource need per test).

Unintuitive scan time

Let us present the problem on example. We will change a little code of our ConditionWatcher sample in order to recreate the problem.

Starting Activity has Button which is not accessible instantly. Button at start is outside the screen and it will be slowly animated in after 1000 milliseconds and animation will last 800 milliseconds.

And the body of IdlingResource used in test, which waits until Button translationY will reach 0 value.

And we started the test. Click was performed by Espresso and next Activity was successfully started. But after that test froze and we had to wait for 30 seconds until we received an error:

android.support.test.espresso.PerformException: Error performing ‘single click — At Coordinates: 539, 1001 and precision: 16, 16’ on view ‘with id: com.azimolabs.f1sherkk.conditionwatcherexample:id/btnStart’.

So we could observe that test went successful but we still received freeze and then error. Let’s look at the logs.

17:14:05.502 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.520 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.520 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.561 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.562 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.571 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.572 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.578 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.578 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.584 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.584 Test-log: isIdleNow() called, conditionStatus = true
17:14:05.588 Test-log: btnStart on click invoked
17:14:05.754 Test-log: ListActivity onCreate()
17:14:09.794 Test-log: isIdleNow() called, conditionStatus = false
17:14:14.800 Test-log: isIdleNow() called, conditionStatus = false
17:14:19.807 Test-log: isIdleNow() called, conditionStatus = false
17:14:24.809 Test-log: isIdleNow() called, conditionStatus = false
17:14:29.816 Test-log: isIdleNow() called, conditionStatus = false
17:14:34.823 Test-log: isIdleNow() called, conditionStatus = false
17:14:39.798 Test-log: isIdleNow() called, conditionStatus = false

Conclusions:

1. When isIdleNow() returns false, the next time it will be called in around 5 seconds.
2. When **isIdleNow() **returns true,it will be called again a few times.
3. When isIdleNow() returns true, call interval will change from 5 seconds to few milliseconds.
4. Method isIdleNow() is still called after perform() or check()method were invoked. If its value will change from true to false, code will be deadlocked until timeout is thrown.

To prevent that, remember to block condition check inside isIdleNow() after it returns true, if its value might change after performed by Espresso action. (like in our case, new Activity was started so Button we were waiting for didn’t exist in new layout).

Boilerplate and readability

If more than one IdlingResource is needed in test case, remember that all IdlingResources are added to same list inside IdlingResourceRegistry which is shared between test. Consequently every registered IdlingResource set should be unregistered before creation of new one.

There might be need to create IdlingResource objects in test in order to unregister them after usage and registering new resources which makes code looks like that:

And now let’s say click() action at line 6 will fail. At line 5 we registered IdlingResource which was added to the list, but because line 6 caused an error — unregisterIdlingResources() at line 7 wasn’t called. If there are more test cases in this java class it will lead to malfunction of whole test set.

Consequently it might become necessary to unregister resources not only inside tests but also between tests e.g:

---

ConditionWatcher

When we started our adventure with Android Espresso, we came across with various problems connected to IdlingResources. Before we were able to understand how Espresso works on lower layer and explain behaviour of IdlingResources, we created our own tool beforehand and based tests on it. As we can see now principle of operation is very similar, yet we would like present perks of ConditionWatcher as they might become useful to you.

ConditionWatcher is a simple class, containing 51 lines of code, created in order to make Android automation testing easier, faster, cleaner and more intuitive. It synchronises operations that might occur on any thread — with test thread. ConditionWatcher can be used as a replacement to Espresso’s IdlingResources or it can work in parallel with them.

It is available at AzimoLabs GitHub.

How does it work?

ConditionWatcher is a singleton which is being created once during whole test process and destroyed with it. Idea is very simple:

Pass Instruction class containing logical expression to ConditionWatcher. Test code will be stopped while passed logical expression returns false and instantly released after expression returns true. Timeout will be thrown if wait takes too long.

Important to state is that ConditionWatcher synchronisation is not connected to Espresso. It performs wait by using **Thread.sleep() **on thread where waitForCondition() was requested (so basically test thread). Thanks to that test author is not bound to any Espresso class or idling process. It is easy to modify ConditionWatcher code and adjust it to any kind of project.

By default ConditionWatcher provides test author with three methods:

1. setWatchInterval() — allows to change interval with which sent, inside Instruction, logical expression is checked. By default set to 250 milliseconds.
2. setTimeoutLimit() — allows to set how much time ConditionWatcher should wait for checkCondition() to return true. By default set to 60 seconds.
3. waitForCondition(Instruction instruction) — takes Instruction containing logical expression as a parameter and calls its checkCondition() method, with currently set interval, until it returns value true. During that time test code won’t proceed to next line. If wait takes too long then Exception is thrown.

From the other side Instruction class happens to have very similar structure to IdlingResource:

1. checkCondition() — a core method which is equivalent to isIdleNow() of IdlingResource. Logical expression which change along with monitored dynamic resource status should be implemented there.
2. getDescription() — is a String returned along with timeout Exception. Test author can place there any kind of log that will contain information helpful during test crash debugging.
3. setDataContainer() and getDataContainer() is a bundle that can be added to Instruction to share primitive types (e.g universal Instruction that waits for any kind of view to become visible can be created, and resId could be sent via bundle).

Good remarks connected to previous paragraph would be that :

• when checkCondition() returns true once it won’t be called again,
• Instructions are not shared between tests or stored anywhere, consequently test author doesn’t need to remember to unregister them and test case becomes more clean,
• it is possible to adjust scan interval time to case need, it will save a lot of time after test set will grow.

When IdlingResource usage was explained in previous paragraphs there was an example of IdlingResource which waits for loading Dialog to disappear from view hierarchy. To compare, the same logic was implemented with usage of Instruction:

And after creation of such Instruction all is need to be done is to call:

ConditionWatcher.waitForCondition(new LoadingDialogInstruction());

Code will stop at this line and it is sure that next one will be called after Dialog has already disappeared. Otherwise test author will receive information about test failure from ConditionWatcher.

Full example is provided in ConditionWatchers sample project.