Commit graph

159 commits

Author SHA1 Message Date
t-m-w
c2f737458c Initial support for backup of D2D-only apps
Allow backup of apps that would otherwise only support device-to-device
migration. This is an initial-support patch to help determine the
viability of this approach.

Known issues / TODO:
* System-scheduled backups will not handle D2D-only apps, unless
  accompanied by a framework change forcing OperationType.MIGRATION.
  Backups triggered by the connection of a USB device or by Seedvault's
  StorageBackupService (files) scheduling are not affected, so they
  *will* back up D2D-only apps as expected; otherwise, the user may
  need to perform a backup manually via Backup Now.
* Apps with `allowBackup="false"` will appear in Backup Status under
  "Installed Apps" rather than "Apps that do not allow data backup",
  and their status will always be blank until they have been backed
  up. If they are not eligible for migration, it will never change.

Other notes:
* The unit test for excluding the Storage Plugin provider from backups
  was discussed, deemed unnecessary, and removed.

Co-authored-by: Oliver Scott <olivercscott@gmail.com>
Change-Id: I5a23d68be66f7d8ed755f2bccb9570ab7be49356
2023-11-29 23:55:40 -05:00
Torsten Grote
cb95444ad4
Bump minSdk to 33 and upgrade unit test dependencies 2023-10-10 15:44:53 -03:00
Torsten Grote
74566b902a
Upgrade kotlin-bip39-jvm to 1.0.6
Bip39ComparisionTest was run before and after upgrading org.bitcoinj:bitcoinj-core to latest version to ensure that kotlin-bip39 is still behaving the same.

A small tweak in WordListTest was needed because WorldList constructor was made internal.
2023-10-10 15:44:52 -03:00
Torsten Grote
87a800438b Always start a new RestoreSet when initializing the device
This avoids deleting the current backup when the user disables backups (or the system decides to do a random re-init).
2023-01-06 20:09:16 +05:30
Torsten Grote
82f23b7800
Set minSdk to 32 and remove code for old SDKs
We can't go to 33 yet, because roboelectric doesn't support that
2022-08-23 11:44:18 -03:00
Aayush Gupta
02af9a27a8 restore: install: Cast any() to Int to avoid oevrload errors
Signed-off-by: Aayush Gupta <aayushgupta219@gmail.com>
2022-08-23 18:36:42 +05:30
Torsten Grote
d72bee523b Mock ApplicationInfo
as it now calls the native SystemClock.uptimeMillis() method which is hard to mock
2022-08-23 18:30:55 +05:30
t-m-w
7901aec408 Exclude storage provider from restore and update affected unit tests,
additionally adding a test to ensure the storage provider is excluded.

Otherwise, when the storage provider (i.e. DAVx5 or Nextcloud) is installed,
it will be terminated, and the restoration process will be interrupted.

Change-Id: Ic1be201b673d718b416009ff61f5d975c89b6a62
2022-07-11 16:58:52 -04:00
Torsten Grote
d598aac81e Clean up system USB storage feature a bit 2022-04-29 23:43:09 +05:30
Torsten Grote
579919d5e7 Code style and lint after Android 12 version bumps 2021-10-13 16:25:41 +05:30
Torsten Grote
b029b0b029 Improve behavior of @pm@ backups when we can't do backups
Before, we were faking a backup and just returned true, but remembering that next time, we need to do a fresh non-incremental @pm@ backup.
Now, we backup to local cache, but don't upload it. On next run, when we can do backups again, we will upload the updated cache. This simplifies things and reduces the special logic required.
2021-10-13 16:15:20 +05:30
Torsten Grote
36c35d6f98 Force backup initialization when we have no cached metadata
as it can happen after clearing app data
2021-10-13 16:15:20 +05:30
Torsten Grote
b8b6882484 Don't try to clear data of NO_DATA system apps
This speeds up the backup quite a bit since there's many such apps
2021-10-13 16:15:20 +05:30
Torsten Grote
79777a7b6f Close K/V DB after restoring records
also don't log stack trace when not finding backup type
2021-10-13 16:15:20 +05:30
Torsten Grote
3c5e4120c7 Refactor Storage Plugin API 2021-10-13 16:15:20 +05:30
Torsten Grote
50066f0317 Let ApkBackup and ApkRestore use the new storage plugin API 2021-10-13 16:15:20 +05:30
Torsten Grote
183e34afd2 Remove legacy backup plugin code 2021-10-13 16:15:20 +05:30
Torsten Grote
1885021c1c Move backup of APKs of opt-out apps to after uploading @pm@ DB 2021-10-13 16:15:20 +05:30
Torsten Grote
a0f3c6b45f K/V restore using single file 2021-10-13 16:15:20 +05:30
Torsten Grote
0c915e5eb8 K/V backup using single file
Tests are still broken until restore has also been implemented with single file approach
2021-10-13 16:15:20 +05:30
Torsten Grote
23bb385190 Use new storage API for full restore 2021-10-13 16:15:20 +05:30
Torsten Grote
674568ca11 Use new storage API for full backups 2021-10-13 16:15:20 +05:30
Torsten Grote
4bdaaa0ce9 Make RestoreCoordinator use the new storage API with salt and backup type
This breaks restores until all the other required changed have been implemented.
2021-10-13 16:15:20 +05:30
Torsten Grote
75cf014e5d Add crypto method to get salted names for package data and APKs
This will only hide installed apps from naive attackers as the APKs are still not encrypted and even then other attacks would be possible.

However, it allows us to simplify our storage plugin API.
2021-10-13 16:15:20 +05:30
Torsten Grote
793663acb5 Add salt and backup type to metadata 2021-10-13 16:15:20 +05:30
Torsten Grote
39cb0c6443 Factor getting secure random bytes into Crypto interface 2021-10-13 16:15:20 +05:30
Torsten Grote
db4103e752 Move code to get available backups from RestorePlugin to BackupPlugin 2021-10-13 16:15:20 +05:30
Torsten Grote
5d1e3debd1 Define new and simpler BackupPlugin API
leaving the old one in place still
2021-10-13 16:15:20 +05:30
Torsten Grote
2932af463c Check version of backup files against expected version from metadata
and throw security exception if it does not match
2021-10-13 16:15:20 +05:30
Torsten Grote
5523e57fe7 Add current metadata to RestoreCoordinator state
so we know which backup version we need to expect during restore
2021-10-13 16:15:20 +05:30
Torsten Grote
aeafc80bb9 Clean up after moving all backup code to new v1 version 2021-10-13 16:15:20 +05:30
Torsten Grote
f4dc776ed3 Do full backups with new version 1 with new crypto
Restoring still supports version 0 with old crypto
2021-10-13 16:15:20 +05:30
Torsten Grote
0c3ea7679b Do K/V backup with new version 1 with new crypto
Restoring still supports version 0 with old crypto
2021-10-13 16:15:20 +05:30
Torsten Grote
3ffb79b04f Write metadata with new version 1
Reading still supports version 0
2021-10-13 16:15:20 +05:30
Torsten Grote
7dceb98670 Add test for decrypting and reading version 0 metadata 2021-10-13 16:15:20 +05:30
Torsten Grote
b707d7f9d0 Add test for full and k/v restore of version 0 backups 2021-10-13 16:15:20 +05:30
Torsten Grote
a5a3a85c6c Add expert settings with an option for unlimited quota
Change-Id: Iebaea41ce4e69912f7cb723bd92e94e4396aa657
2021-09-28 22:02:04 +05:30
Torsten Grote
d6a95e40ec Compares kotlin-bip39 library with bitcoinj library
to ensure that kotlin-bip39 is not malicious and can be upgraded safely
2021-09-24 00:02:43 +05:30
Torsten Grote
05640ebb63 Replace novacrypt BIP39 library with the zcash one
which is MIT licensed instead of GPLv3

Change-Id: I30041de5ab1c1f8c7d5f57d6c60e28300a285259
2021-07-05 23:50:18 +05:30
Torsten Grote
6c633b70c3 Add storage library (and demo app)
and use for periodic files backup
2021-07-05 20:08:29 +05:30
Torsten Grote
851407037e
Store main key for key derivations from 512-bit BIP39 recovery code
This main key will be used later to derive sub-keys for other crypto operations.
2021-02-15 12:16:27 -03:00
Torsten Grote
2cd2f73241
Use a TestApp for UnitTests so we can use different modules for injection 2021-02-15 12:16:27 -03:00
Torsten Grote
b1a0c1b2e2
Group app status list into three sections
* important system apps
* user apps
* apps not allowing backup
2020-11-18 16:35:45 -03:00
Torsten Grote
e2f0d19f77
Pretend to make successful @pm@ backup when no backup is possible
This is the same behavior as Google backup when it has no internet connection and after extensive research the only option we can keep the system from considering the backup state to be compromised.

K/V backups are run at least every day, no matter what backup interval we set in settings and when they run, we don't get asked before, if now is a good time for backups. So we need to fake an OK for @pm@ backup and can error out afterwards without compromising state.
2020-10-23 07:33:59 -03:00
Torsten Grote
141fe7575d
Check also availability of internet access when using online storage
This moves these availability checks into the Storage class, so they can be used in various places without duplicating code.
2020-10-23 07:33:59 -03:00
Torsten Grote
7401ead553
Request backoff when asked to backup to network storage while no internet available
K/V backups are normally only attempted when charging and having an (un-metered) internet connection. However, if the system could not do a backup for more than a day, it ignores these requirements and still attempts a backup run. If a backup storage is used that is only accessible on the internet, but there is no internet connection, the backup attempt will fail. Therefore, we check if our storage requires the internet and if so, we treat it similar to a removable storage, by rejecting backup attempts and suppressing error notifications.
2020-10-23 07:33:58 -03:00
Torsten Grote
2cde417c8c
Add config option to re-install apps with feature modules only on same device
There is a possibility that incompatible APK splits make a an app crash when starting after re-installing it.
With that config option each OEM can decide with they want to take this risk or not.
2020-10-22 08:00:55 -03:00
Torsten Grote
0f0f198228
Add unit tests for re-installing apps with APK splits 2020-10-13 16:19:05 -03:00
Torsten Grote
608e67cb65
Refactor existing ApkRestore unit tests to make adding new ones easier 2020-10-13 16:19:05 -03:00
Torsten Grote
b3db859b40
Re-install APK splits if they are compatible and have proper hash 2020-10-13 16:07:19 -03:00
Torsten Grote
68a6403c4b
Add a compatibility checker for APK splits
that tries to figure out compatibility only based on the name of the split.
This is not an exact science and there might be errors, but we hope to correctly identify most cases that matter in practice.
2020-10-13 16:07:19 -03:00
Torsten Grote
643247b600 Change UI for re-installation of system apps
We are re-installing system apps if they are present on the restore device as a system app and have a newer version code.
Before, when one of those conditions is not true, we were showing a failure and gave the user the option to re-install the app from an app store.
Now, we don't offer the manual re-install option anymore and only show a success when a newer or same version of the system app is already installed.
2020-10-13 18:32:04 +05:30
Torsten Grote
0971c5db19 Do not back up APKs of test-only apps, as we can not re-install them anyway
The flag to allow installation gets filtered out after we set it:
http://aosp.opersys.com/xref/android-11.0.0_r5/xref/frameworks/base/services/core/java/com/android/server/pm/PackageInstallerService.java#544
2020-10-13 18:32:04 +05:30
Torsten Grote
0a8a286826 Update state of opt-out apps, even if they never had any state 2020-10-13 18:32:04 +05:30
Torsten Grote
d6cb34c211 Allow the user to manually re-install apps before data restore starts
When one or more apps fail to install, the user is shown a dialog explaining that we need the apps installed in order for restore to work.
After the dialog is dismissed, the list of apps is resorted so failed apps are at the top. They are made clickable and the user is brought to an app store to re-install them.
2020-10-13 18:32:04 +05:30
Torsten Grote
747384fb59 Refactor InstallResult to be more extensible 2020-10-13 18:32:04 +05:30
Torsten Grote
f45411d81b Refactor code related to APK installs as preparation for upcoming changes 2020-10-13 18:32:04 +05:30
Torsten Grote
3a31e09a04 Back up split APKs as well and store them in the metadata
This will enable us to check compatibility of the splits with the restore device and if compatible, re-install them.
2020-10-13 18:32:04 +05:30
Torsten Grote
af2bf4f60a Support APK splits in metadata 2020-10-13 18:32:04 +05:30
Torsten Grote
1d2c74bf2c Ensure that metadata cache streams get closed 2020-10-08 19:27:51 +05:30
Torsten Grote
6c531066e7 Enable automatic coding style linting with ktlint (also on CI)
This way the coding style is guaranteed to stay consistent.
2020-10-07 19:44:05 +05:30
Torsten Grote
53937bda2f Auto-format code style of all files to match official style
This also adds a note to the README
and the Android Studio coding style files.
2020-10-07 19:44:05 +05:30
Torsten Grote
57f404281c Fix lint warnings 2020-10-07 19:01:39 +05:30
Torsten Grote
cfcf7b35ba Clean up BIP39 dependencies 2020-10-07 19:01:39 +05:30
Torsten Grote
8bdbe6d681 Handle new FLAG_DATA_NOT_CHANGED for K/V backups 2020-10-07 19:01:39 +05:30
Torsten Grote
45ac8883a0 Fix roboelectric unit tests 2020-10-07 19:01:39 +05:30
Torsten Grote
77550a9860 Treat stopped apps different from opt-out apps
Apps that have FLAG_STOPPED will not get backed up, just like apps
without flag ALLOW_BACKUP will not get backed up.
In the UI both cases are shown the same way: app does not allow backup
This can be confusing for the user as it is not true for stopped apps.
Therefore, this commit introduces a new stopped state for apps,
so we can differentiate between both cases.
2020-09-25 05:55:40 +05:30
Torsten Grote
9f2b56e4ce Fix bug where we could not do two subsequent restores
This probably never showed in practice, but it can be triggered easily
when testing with `adb shell bmgr restore`.
2020-09-18 00:22:45 +05:30
Torsten Grote
42ab8ffba5 Cache folder contents in K/V backup/restore
This speeds up things significantly and was needed due to poor
performance of call log backup.
2020-09-17 06:06:14 +05:30
Torsten Grote
1b9a4feddd Clean up backup transport initialization logic
This commit makes creating new RestoreSets explicit.
Initializing a backup transport now actually cleans its data as the AOSP
documentation demands. This should be fine as we usually do a fresh
backup after a new initialization.
Contrary to before, an initialization does not create new RestoreSets
anymore, but works within the existing set. For now, only manually
choosing a new storage location creates a new RestoreSet.
2020-09-11 19:04:38 +05:30
Torsten Grote
80187c8c70 Show percentages in progress notification and x of n status at the end
Fine-grained progress reporting causes apps to show up twice which is
confusing. Also @pm@ metadata and opt-out APKs are too much detail for
normal users. So we decided to only show a percentage in the progress
notification.

When the backup finished, the app now shows "x of n apps backed up"
which is more positive when the previous negative message of how many
apps were not backed up.

Some further minor tweets were done to app counting to report proper
totals.
2020-09-11 19:04:38 +05:30
Torsten Grote
983f917391 Let backup notification report more fine-grained progress
This adds @pm@ record backup and APK backup of opt-out apps to the
progress reporting since these two operations are slow when using a
cloud storage SAF backend.
2020-09-11 19:04:38 +05:30
Torsten Grote
e7a13fdb5c Ensure streams get closed eventually 2020-09-11 19:04:38 +05:30
Torsten Grote
7fdefda85f Replace all instances of DocumentFile#findFile with #findFileBlocking
Also start sticking closer to the official Kotlin formatting style
2020-09-11 19:04:38 +05:30
Torsten Grote
f7df78d2f3 Check for loading cursor also when checking if files exist
Loading cursors can happen with cloud-based documents providers
such as Nextcloud.
When they return a cursor that is still loading,
we might continue with stale information.
So now we wait for a loading cursor to be fully loaded
before continuing.
2020-09-11 19:04:38 +05:30
Torsten Grote
324da2a9e9
Allow the user to exclude apps from backup
Closes #70
2020-01-20 11:58:38 -03:00
Torsten Grote
a98364efbe
Back up APKs as early as possible to not miss any
Fixes #68
2020-01-18 16:42:26 -03:00
Torsten Grote
de16032905
Don't use wildcard imports, because they are considered harmful 2020-01-14 15:40:56 -03:00
Torsten Grote
2bcf82d607
Show heads-up notification when auto-restore fails due to removed storage 2020-01-14 10:17:38 -03:00
Torsten Grote
74183d40d6
Add system app flag to metadata and check before installing system apps
that they are present as an older system app on the restore system.

Also ignore system apps without data to backup.
2020-01-14 10:11:17 -03:00
Torsten Grote
fea53a759f
Also back-up APKs of apps that are not allowed to have their data backed up 2020-01-14 07:45:41 -03:00
Torsten Grote
3d296e1335
Also back up APKs of apps that have no data or are above quota
This should also affect apps that have other errors during the backup
process, but it does not affect apps that opt-out of backup completely.

First part of #65
2020-01-14 07:45:41 -03:00
Torsten Grote
9f01d09962
Don't get or write to full backup output stream before we are not sure there will be data to write 2020-01-14 07:45:39 -03:00
Torsten Grote
690017c050
Only back up APK and write metadata when app was actually backed up
Apps that have nothing to back up start a backup but then get a call to cancelFullBackup()
and never even call finishBackup().
Do not write metadata for such apps, the call got moved to finishBackup().
2020-01-14 07:45:39 -03:00
Torsten Grote
2f352fe828
Show list of packages that we could restore data for 2020-01-14 07:45:39 -03:00
Torsten Grote
7605762631
Re-install backed-up APKs before restoring from backup 2020-01-14 07:45:37 -03:00
Torsten Grote
569e3db385
Fix device initialization and generation of new backup tokens 2020-01-14 07:45:37 -03:00
Torsten Grote
81c2031ce7
Back up APKs to storage (when they changed) and save metadata about them 2020-01-14 07:45:37 -03:00
Torsten Grote
b9cac5ea87
Introduce MetadataManager to handle all metadata related to backups
This now updates the metadata on remote storage and internal cache
after each successful package backup.
2020-01-14 07:45:36 -03:00
Torsten Grote
e1d55c8a4e
Add information about packages to backup metadata
This will be needed when backing up APKs.

ATTENTION: This is a breaking change, we only do because the app hasn't
been released.
2020-01-14 07:45:36 -03:00
Steve Soltys
01098a4d97
Merge pull request #60 from grote/check-messages
Encrypt values of key/value backups with multiple segments if needed
2019-12-22 20:24:33 -05:00
Torsten Grote
58a8f29b51
Encrypt values of key/value backups with multiple segments if needed
This turned out to be necessary, because some values on production
devices are exceeding the maximum segment size.
2019-12-19 08:55:57 -03:00
Torsten Grote
94c7663daf
Use dependency injection with Koin 2019-12-17 09:56:45 -03:00
Torsten Grote
440491425a
Work around DocumentFile bug happening with cloud-based DocumentsProviders
These might return outdated or now content when queried,
then check their cloud storage and report back with up-to-date content.
We now detect this (when looking for backups on newly setup storage)
and wait until the content has been loaded before acting on the
response.

This is affecting and was tested with NextCloud.
2019-12-16 09:11:52 -03:00
Steve Soltys
32f558faf1 Rebrand application as 'Seedvault' 2019-10-28 21:57:47 -04:00
Torsten Grote
470b5a2ccf
Tolerate backup attempts when flash drive is not plugged in
Also remove hardcoding of PACKAGE_MANAGER_SENTINEL constant
2019-09-23 10:29:01 -03:00
Torsten Grote
007dd7759d
Save the time of the last backup and only do automatic flash drive backups once a day
This commit also turns SettingsManager into a class, so we can mock
and later also inject it.
2019-09-23 10:10:27 -03:00
Torsten Grote
650642068e
Don't try to do backups if storage is not available 2019-09-23 10:10:26 -03:00
Torsten Grote
10ad6d6b2d
Improve error message when no backups could be found for restore 2019-09-17 12:14:53 -03:00
Torsten Grote
af43c6154d
Get rid of device folders, use unix epoch as backup token and store it 2019-09-11 15:26:10 -03:00