1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
|
# Profcollect
Profcollect is a system daemon that facilitates sampling profile collection and reporting for native
platform applications.
Profcollect can only be enabled on `userdebug` or `eng` builds.
## Supported Platforms
Currently Profcollect only supports collecting profiles from Coresight ETM enabled ARM devices.
Instructions to enable Coresight ETM can be found from the
[simpleperf manual](https://android.googlesource.com/platform/system/extras/+/refs/heads/master/simpleperf/doc/collect_etm_data_for_autofdo.md).
## Usage
Profcollect has two components: `profcollectd`, the system daemon, and `profcollectctl`, the command
line interface.
### Collection
`profcollectd` can be started from `adb` directly (under root), or automatically on system boot by
setting system property through:
```
adb shell device_config put profcollect_native_boot enabled true
```
Profcollect collects profiles periodically, as well as through triggers like app launch events. Only
a percentage of these events result in a profile collection to avoid using too much resource, these
are controlled by the following configurations:
| Event | Config |
|------------|------------------------|
| Periodic | collection\_interval |
| App launch | applaunch\_trace\_freq |
Setting the frequency value to `0` disables collection for the corresponding event.
#### Custom configuration
In adb root:
```
# Record every 60s (By default, record every 10m). The actual interval will be longer than the
# set value if the device goes to hibernation.
oriole:/ # setprop persist.device_config.profcollect_native_boot.collection_interval 60
# Each time recording, record ETM data for 1s (By default, it's 0.5s).
oriole:/ # setprop persist.device_config.profcollect_native_boot.sampling_period 1000
# Set ETM data storage limit to 50G (By default, it is 512M).
oriole:/ # setprop persist.device_config.profcollect_native_boot.max_trace_limit 53687091200
# Enable ETM data collection (By default, it's decided by the server).
oriole:/ # setprop persist.device_config.profcollect_native_boot.enabled true
# After adjusting configuration, need to restart profcollectd
oriole:/ # setprop ctl.stop profcollectd
# Wait for a few seconds.
oriole:/ # setprop ctl.start profcollectd
# Check if profcollectd is running
oriole:/ # ps -e | grep profcollectd
root 918 1 10945660 47040 binder_wait_for_work 0 S profcollectd
# Check if the new configuration takes effect.
oriole:/ # cat /data/misc/profcollectd/output/config.json
{"version":1,"node_id":[189,15,145,225,97,167],"build_fingerprint":"google/oriole/oriole:Tiramisu/TP1A.220223.002/8211650:userdebug/dev-keys","collection_interval":{"secs":60,"nanos":0},"sampling_period":{"secs":1,"nanos":0},"binary_filter":"^/(system|apex/.+)/(bin|lib|lib64)/.+","max_trace_limit":53687091200}
```
To check existing collected ETM data:
```
oriole:/ # cd data/misc/profcollectd/trace/
oriole:/data/misc/profcollectd/trace # ls
```
To check if ETM data can be collected successfully:
```
# Trigger one collection manually.
oriole:/ # profcollectctl once
Trace once
# Check trace directory to see if there is a recent manual trace file.
oriole:/ # ls /data/misc/profcollectd/trace/
20220224-222946_manual.etmtrace
```
If there are too many trace files, we need to processing them to avoid reaching storage limit.
It may take a long time.
```
oriole:/ # profcollectctl process
Processing traces
```
### Processing
The raw tracing data needs to be combined with the original binary to create the AutoFDO branch
list. This is a costly process, thus it is done separately from the profile collection. Profcollect
attempts to process all the traces when the device is idle and connected to a power supply. It can
also be initiated by running:
```
adb shell profcollectctl process
```
### Reporting
#### Manual
After actively using the device for a period of time, the device should have gathered enough data to
generate a good quality PGO profile that represents typical system usage. Run the following command
to create a profile report:
```
$ adb shell profcollectctl report
Creating profile report
Report created at: 12345678-0000-abcd-8000-12345678abcd
```
You can then fetch the report by running (under root):
```
adb pull /data/misc/profcollectd/report/12345678-0000-abcd-8000-12345678abcd.zip
```
#### Automated Uploading to Server
*In development*
### Post Processing
For each trace file, run:
```
simpleperf inject \
-i {TRACE_FILE_NAME} \
-o {OUTPUT_FILE_NAME}.data \
--binary {BINARY_NAME} \
--symdir out/target/product/{PRODUCT_NAME}/symbols
```
Afterwards, run [AutoFDO](https://github.com/google/autofdo) to generate Clang PGO profiles:
```
create_llvm_prof \
--profiler text \
--binary=${BINARY_PATH} \
--profile=${INPUT_FILE_NAME} \
--out={OUTPUT_FILE_NAME}.profdata
```
Finally, merge all the PGO profiles into one profile:
```
find {INPUT_DIR} -name *.profdata > proflist
prebuilts/clang/host/linux-x86/llvm-binutils-stable/llvm-profdata merge \
--binary \
--sample \
--input-files proflist \
--output merged.profdata
```
More profile data usually generates better quality profiles. You may combine data from multiple
devices running the same build to improve profile quality, and/or reduce the performance impact for
each device (by reducing collection frequency).
|
