Android S 輸入法相關新特性整理

語言: CN / TW / HK

Android S 輸入法相關新特性整理

1.首先在S上,對輸入法的target窗口有了更準確的命名

具體如下:

| R版本 | S版本 | 定義 | | :---: | :---: | :--: | InputMethodTarget |mImeLayeringTarget|輸入法放置的窗口 | InputMethodInputTarget | mImeInputTarget | 輸入法輸入的目標窗口 | InputMethodControlTarget | mImeControlTarget | 輸入法的控制窗口 |

2.然後谷歌主要通過6筆change來實現更好的輸入法 transition體驗

其主要思想是通過保存輸入法的snapshot ,來保證輸入法在進入最近任務界面以及在app的transition過程中能夠始終保證可見性,從而提高更好的使用體驗。

2.1 首先第一筆change主要有以下幾筆修改:

``` commit 60222bf60db8e7ccfa9d025ab1b24d7f7e31eea9 Author: Ming-Shin Lu [email protected] Date: Mon Sep 14 00:07:04 2020 +0800

Better IME transition while switching app with recents (1/N)

Changes includes: - In DC#computeImeTarget, keeps the IME target with the last window while swiping up to recents to prevent flickering due to IME hide animation on top of recents. - Screenshot Task with IME window if IME is attatched to the app. - Ignore hiding current IME if IME is attached to the app. ```

  • DisplayContent#computeImeTarget方法主要是在window層級變化時重新計算返回當前的輸入法target窗口,第一筆change在該方法內添加邏輯判斷,當新的target窗口是最近任務窗口同時之前的target窗口的輸入法是可見的,那麼直接返回之前的target窗口,無需再計算新的target窗口,等待需要顯示輸入法或者重新relayout時再計算輸入法target窗口

  • RecentsAnimationController#hideCurrentInputMethod方法內添加邏輯,首先先調用updateImeParent方法保證ImeParent始終正確,然後就是判斷當前輸入法有沒有attach到app上,如果輸入法當前已經attach到app上,那麼在進入最近任務界面時就不執行其隱藏動畫

TaskSnapShotController#createTaskSnapshot方法,在添加excludeLayer時添加判斷條件,原先只要輸入法窗口不為null,就把輸入法的surface添加到excludelayers內,現在是同時當輸入法沒有attach到app上時,才將其添加到excludelayers內。換句話説,如果當前輸入法attach到app上時,創建snapShot的時候是可以將其顯示出來的。

2.2 第二筆change主要修改如下:

``` commit 73aab1dc1fce55a544f5ecffe1c4cb7939d77be9 Author: Ming-Shin Lu [email protected] Date: Mon Sep 14 12:35:41 2020 +0800

Better IME transition while switching app with recents (2/N)

In TaskSnapshotController#handleClosingApps will capture task snapshot
while applying closing animation or during screen turning-off.

this will overwrite the captured task from RecentsAnimationController,
and makes while quick switching tasks, sometimes will not see the IME
surface on the task snapshot.

Add a check in places which triggers task snapshot to ignore addidnal
snapshot request while RecentsAnimation is active.

Also refined Task#isTaskAnimating as isAnimatingByRecents to simplify
the logic and add the test for its behavior.

Bug: 166736352
Test: manual by:
      1) Tapping EditText in app and make keyboard shown
      2) Quick switch back and force between apps by swiping navbar
      3) Verify if the task snapshot with IME persists shown during
         switching apps
Test: atest RecentsAnimationControllerTest#testIsAnimatingByRecents

```

TaskSnapShotController#getClosingTasks內添加邏輯判斷,如果某task是正在執行最近任務動畫,那麼在mSkipClosingAppSnapshotTasks內添加該task。因為RecentsAnimation已經在處理task snapshot,這裏就沒必要處理由RecentsAnimation控制的task了。

2.3 第三筆change主要修改如下:

概括:通過對InsetsSourceControl設置skipAnimation值 ,(由於next task已經存在Ime surface)保證Insetscontroller不會重新應用輸入法顯示動畫

``` commit d579c94480fa98042dd38721de0e3a18fec67e9e Author: Ming-Shin Lu [email protected] Date: Mon Oct 12 22:10:44 2020 +0800

Better IME transition while switching app with recents (3/N)

- Add hasImeWindowSurface in TaskSnapShot
- Add InsetsSourceControl#getAndClearSkipAnimationOnce for skiping IME showing animation once when starting window with IME surface

通過對InsetsSourceControl設置skipAnimation值 ,(由於next task已經存在Ime surface)保證Insetscontroller不會重新應用輸入法顯示動畫

Test: manual as below steps
    0) Enabling developer options -> Quick settings developer tiles ->
       Window animation scale to slow down transition animation.
    1) Launch an app with focusing an editor to show soft-input
    2) Swipe out app task to back to launcher
    3) Using quick switch or taping shortcut to bring back the app task
    4) Verify that should be no IME showing animation happens during task transition.
Change-Id: I83ffc03119e01da71ad12f8ad8043cf0730dfd50

```

  • TaskSnapShot內添加hasImeSurface值來判斷當前snapShot是否存在ime的surface
  • InsetsSourceControl內添加mSkipAnimationOnce值以及相關邏輯,InsetsController#applyAnimation

提供布爾值判斷當前動畫是否需要跳過(比如當前app內顯示了輸入法,按任務健進入最近任務時,此時輸入法的隱藏動畫就需要skip)

  • ImeInsetsSourceProvider#getControl方法,在這裏獲取InsetsSourceControl時,設置該control的mSkipAnimationOnce值,(主要依據條件就是target窗口的snapshot內是否存在ImeSurface)

  • InsetsController#applyAnimation,在該方法內如果當前Insets為輸入法時,獲取其InsetsSourceControl的mSkipAnimationOnce值,並在構造InternalAnimationControlListener時將skipAnim值傳遞到mDisable變量,通過這個值來控制動畫是否enable

2.4 (重點)第四筆change主要修改如下:

概括: 從closing app transit到新的task時,通過showIme的snapshot始終保持輸入法的可見性,而在動畫結束的時候remove掉輸入法的surface

``` commit 4964839ec3b3648ef64782a7005f6ec18cd85548 Author: Ming-Shin Lu [email protected] Date: Fri Oct 16 01:23:55 2020 +0800

Better IME transition while switching app with recents (4/N)

This CL introduces TaskSnapshotController#snapshotImeFromAttachedTask to
attach IME screenshot when performing closing transition.

This can improve app transition without jank or flickering while the IME
insets control transits from closing task to the next task, we keep the
IME surface visibility by placing the IME screenshot with calling
DC#showImeScreenshot to be a part of task while animating
app transition, and then remove it with DC#removeImeScreenshotIfNeeded
when the transition animation finished or no longer used gracefully.

Test: manual as below steps:
   1) Launch an app with focusing an editor (e.g. Dialer)
   2) Swipe down status bar and tap Settings icon.
   3) Verify that when doing task transition animation, app activity
      with IME keeps visible.
Test: manual as below steps:
   1) With 2-button or 3-button gesture, launch an app with focusing an
      editor to show soft-keyboard.
   2) Pressing home key
   3) Verify the IME screenshot keeps visible and animates with closing
      transition smoothly.

```

  • DisplayContent#setImeLayingTarget方法內添加調用attachAndShowImeScreenshotOnTarget的邏輯,在設置新的或者null ImeLayingTarget的時候,這時候為了保證優雅過渡,需要保持原先的ime surface,因此在這裏添加調用show imeSurface的方法 (這裏是創建ime surface的其中一種調用方式)

  • DisplayContent#attachAndShowImeScreenshotOnTarget,這裏在原先的target窗口內創建ime surface

``` void attachAndShowImeScreenshotOnTarget() { // No need to attach screenshot if the IME target not exists or screen is off. if (!isImeAttachedToApp() || !mWmService.mPolicy.isScreenOn()) { // 滿足不需要生成ime surface的條件 直接return return; }

final SurfaceControl.Transaction t = getPendingTransaction();
// Prepare IME screenshot for the target if it allows to attach into.
if (mInputMethodWindow != null && mInputMethodWindow.isVisible()) {
    final Task task = mImeLayeringTarget.getTask();
    // Re-new the IME screenshot when it does not exist or the size changed.
    final boolean renewImeSurface = mImeScreenshot == null
            || mImeScreenshot.getWidth() != mInputMethodWindow.getFrame().width()
            || mImeScreenshot.getHeight() != mInputMethodWindow.getFrame().height();
    if (task != null && !task.isHomeOrRecentsRootTask()) {
        SurfaceControl.ScreenshotHardwareBuffer imeBuffer = renewImeSurface
                ? mWmService.mTaskSnapshotController.snapshotImeFromAttachedTask(task)
                : null;
        // 如果原先已存在ime surface 這裏先remove掉 然後重新生成        
        if (imeBuffer != null) {
            // Remove the last IME surface when the surface needs to renew.
            removeImeSurfaceImmediately();
            mImeScreenshot = createImeSurface(imeBuffer, t);
        }
    }
}

final boolean isValidSnapshot = mImeScreenshot != null && mImeScreenshot.isValid();
// Showing the IME screenshot if the target has already in app transition stage.
// Note that if the current IME insets is not showing, no need to show IME screenshot
// to reflect the true IME insets visibility and the app task layout as possible.
if (isValidSnapshot && getInsetsStateController().getImeSourceProvider().isImeShowing()) {
    if (DEBUG_INPUT_METHOD) {
        Slog.d(TAG, "show IME snapshot, ime target=" + mImeLayeringTarget);
    }
    t.show(mImeScreenshot);
} else if (!isValidSnapshot) {
    removeImeSurfaceImmediately();
}

} ```

  • DisplayContent#createImeSurface 該方法主要就是創建ime的SurfaceControl

SurfaceControl createImeSurface(SurfaceControl.ScreenshotHardwareBuffer imeBuffer, Transaction t) { final HardwareBuffer buffer = imeBuffer.getHardwareBuffer(); if (DEBUG_INPUT_METHOD) Slog.d(TAG, "create IME snapshot for " + mImeLayeringTarget + ", buff width=" + buffer.getWidth() + ", height=" + buffer.getHeight()); final ActivityRecord activity = mImeLayeringTarget.mActivityRecord; final SurfaceControl imeSurface = mWmService.mSurfaceControlFactory.apply(null) .setName("IME-snapshot-surface") .setBufferSize(buffer.getWidth(), buffer.getHeight()) .setFormat(buffer.getFormat()) .setParent(activity.getSurfaceControl()) .setCallsite("DisplayContent.attachAndShowImeScreenshotOnTarget") .build(); // Make IME snapshot as trusted overlay InputMonitor.setTrustedOverlayInputInfo(imeSurface, t, getDisplayId(), "IME-snapshot-surface"); Surface surface = mWmService.mSurfaceFactory.get(); surface.copyFrom(imeSurface); surface.attachAndQueueBufferWithColorSpace(buffer, null); surface.release(); t.setRelativeLayer(imeSurface, activity.getSurfaceControl(), 1); t.setPosition(imeSurface, mInputMethodWindow.getDisplayFrame().left, mInputMethodWindow.getDisplayFrame().top); return imeSurface; }

DisplayContent#showImeScreentShot、 removeImeScreenshotIfPossible、removeImeSurfaceImmediately 、onWindowAnimationFinished

這些都是 顯示和移除Ime surface的相關方法

showImeScreentShot是在app transition是或者swipe到最近任務時調用(WindowContainer內應用動畫時調用該方法)

removeIme是在動畫結束是或者ime surface淘汰時如旋轉屏引起size變換

  • WindowContainer#applyAnimationUnchecked 在顯示跳轉動畫時,這裏調用顯示 ime的surface

  • WindowContainer#doAnimationFinished 在動畫結束時 回調dc的onWindowAnimationFinished方法(該方法會remove掉 ime的surface)

2.5 第五筆change主要修改如下:

概括: 在updateImeControltarget的時候調用updateImeParent,保證此時輸入法的可見性已經確定了,這樣可以防止閃爍問題,然後在執行進入最近任務動畫過程中,再次check ime parent以及ime是否正確

``` commit d4d90ac8a88e601a4418c44fe45563070581c19b Author: Ming-Shin Lu [email protected] Date: Wed Nov 11 13:17:18 2020 +0800

Better IME transition while switching app with recents (5/N)   在updateImeControltarget的時候調用updateImeParent,保證此時輸入法的可見性已經確定了,這樣可以防止閃爍問題、更改InputMonitor防止錯按、在旋轉動畫時保證imeParent的正確性

With CL[1], the IME surface will have snapshot when transtioning to the
next task.

We can now remove the previous hacky logics like dedicates to keep the
previous IME and make the true IME target while task transitioning.
And, move the call of updateImeParent() from
DC#setInputMethodTarget to DC#updateImeControlTarget,

to ensure that the reparenting of IME insets source control can be done
when the IME insets visiblity settled down after the IME insets control
changed, to prevent unnecessary flickering during the time period between
reparenting IME parent and start IME insets animation.

Also, modify UpdateInputForAllWindowsConsumer#accept to let
mRecentsAnimationInputConsumer can be above IME target activity which to
prevent mis-touch or keystoke may left while quickly taping
soft-keyboard during swping up to recents.

```

  • DisplayContent#updateImeControlTarget 在該方法內更新ime parent,這個是最佳的時機,

(在setImeLayeringTargetInner方法內,原先是在updateImeControlTarget方法之前調用updateImeParent)

  • RecentsAnimationController#setAnimationTargetsBehindSystemBars,在執行進入最近任務界面動畫時,首先更新ime parent保證正確的ime parent,然後再次確認ime是否attach到之前的task上,如果沒有,執行隱藏輸入法操作

2.5 第六筆change主要修改如下:

概括: 通過設置FrozenInsetsState 來保證正在執行closing的window不會收到非期望的insets變化(比如新window的insets的分發)(也就是InsetsStateController會根據是否存在freeze的InsetsState來判斷是否分發insetsChanged),同時添加freeze以及unFreeze Insetsstate的操作

``` commit 0f536fa99489c5cfd2d266e94798b19540e2cd12 Author: Ming-Shin Lu [email protected] Date: Thu Jan 14 21:03:15 2021 +0800

Better IME transition while switching app with recents (6/N) 
Add WindowState#{freezeInsetsState, clearFrozenInsetsState} to freeze
the insets state of the window to keep the insets state when the window
is in app exiting transition, to ensure the exiting window won't
receive unexpected insets changes from the next window.

And, in order to easy maintain the logic of dispatching all windows's
insets changed event in InsetsStateController, instead of adding
complicated rule in InsetsStateController to judge if the window can
disatch insets change, we can replace with
WindowState#isReadyToDispatchInsetsState() to wrap the dispatch
judgement.

Bug: 166736352
Test: atest CtsWindowManagerDeviceTestCases WmTests
Test: atest WindowStateTests#testSetFreezeInsetsState
Test: atest WindowContainerTests#testFreezeInsetsStateWhenAppTransition

Change-Id: I56418733c1abbd73e88a8918a5d55ecc15344c5e

```

  • WindowState內添加mFrozenInsetsState的定義以及相關方法,意為凍結insets的state,在一些情況下沒有必要與client端保證同步,例如在退出動畫時

isReadyToDispatchInsetsState方法返回代表是否準備分發insetsState,如果不存在freeze的InsetsState,即可分發

  • DisplayContent#onWindowAnimationFinished,在動畫結束時清除所有的freeze的insetsState

  • InsesStateController 修改mDispatchInsetsChanged 這個consumer,保證分發時不存在freeze的InsetsState

總結

關於S上輸入法的新特性大概就總結到這裏了,S上的輸入法在動畫切換以及app transition的時候更好的提高了用户的體驗度。 如果大家有何疑問或者建議,歡迎評論留言~ 與大家一起共同進步!~