Wednesday, August 21, 2019

iOS 12 Notifications Triage Parser

Short Version

Introducing a Python 3 script that looks for the UserNotifications folder in iOS 12 full file system extractions and parses the iOS notifications to easily triage their content. The script detailed below is a technical application of the research done at d204n6.com by my friend Christopher Vance that he kindly shared with me before making it public. Check out his blog on the topic at:
https://blog.d204n6.com/2019/08/ios-12-delivered-notifications-and-new.html
Script download:
https://github.com/abrignoni/iOS-Notifications-Parser
Script purpose:

  1. To parse the iOS notifications plists for every app that has the functionality enabled. 
  2. Make a report of the plist contents giving the user the ability of hiding non-human readable, repeatable, and well know data by pressing a button on the HTML report.
  3. Report on and export any incepted bplists (full bplists found within a plist) for further analysis.
Script reason:
  1. As stated in d204n6.com there can be a wealth of data in the iOS screen notifications to include snippets of user generated content like chat messages, images received, and distinct alerts that might not be accessible in other ways.
Pre-requisites
  1. Python 3.
  2. Full file system extraction of an iOS 12 device or the UserNotifications directory. If extracting the directory itself for processing be aware that the script depends on the UserNotification directory (where notifications on iOS are kept) to be at least one level down (or more) from the data directory provided to the script.

Long Version 

When Chris shared his latest research with me I was immediately impressed on how much relevant data is contained in iOS notifications. For further details his blog post above is required reading. In this post I will only go into how to use the script to triage these important plists that seem to be overlooked but shouldn't.

Script usage

After downloading the script and configurations files you should see 4 files in the image.

Scripts and configuration files.

  1. ccl_bplist.py

    Used to deserialize NSKeyedArchiver bplists. Thanks to Alex Caithness came up with this module. It saves us a lot of headaches. Added his module to my repo for convenience. It can be downloaded directly from the source here: https://github.com/cclgroupltd/ccl-bplist
  2. iOSNotificatonsParser.py

    This is where the magic happens. It searches in a specified directory for the UserNotifications directory and when found parses the DelivereNotifications.plist for every app that has notification data.
  3. NotificationsParams.txt

    It contains strings that I consider to be common, unreadable, or repetitive. The items in the list (one per line) are used to determine if they are to be hidden, not eliminated, from the final report. Be aware that the final HTML report has a couple of buttons that allow you to hide or show those fields as needed. To add more string to hide just add a new line to the text file. One string per line.
  4. script.txt

    Contains the javascript necessary to enable the hide/show functionality in the HTML report. It gets added to each report at processing time.
Usage.

The script only has one parameter, the data directory to be parsed. See the help below.

When the script runs it tells you what notification is parsing and if a bplist was found within the plist. If found it will tell you that it was exported.


See the highlighted section above that shows a bplist exported. When done it also advises how many plist were processed, how many exported bplist, and how long processing took.


After the script runs a report directory will be created in the same location where the script resides.


As seen the report directories are timestamps so the script can be run multiple times and each time it will generate a new report directory. Within the directory each app has its own unique directory named after the app's bundle id.


Each app will have a report and exported bplist if any exists. For the screen time notifications in this data set one sees the following:

Each HTML report has a header and the Hide/Show buttons on the top.


Let's zoom in a little on the buttons.

As Christopher explains some of the ASCII values might not be important, unreadable, too much, or are simply repetitive. Hide rows hides them as explained previously by referencing the content of the NotificationsParams.txt file.

It will go from tons of pages to something like the following:

This is the same report. It has hidden a lot of repetitive data. Important note. It is worthwhile to always look at the full report if the app is important to the case. The report is only for triage purposes and will always require validation after execution. This is even more true when talking about the contents of NS.data within a plist. In some cases is either data that is not relevant or unreadable. In many cases it can contain a full bplist in them. The report deserializes this data and lets you read it. It is hard to read due to a lack of proper formatting but at least it will let you know if further analysis is warranted. Here is how a bplist in an NS.data field would look on the report.


Yes, hard to read but still it can be read. If anything pertinent is found then go and take the exported bplist and use any viewer for further and proper analysis.


Here is an example of the exported bplist and how a third party viewer shows you the data with ease.

Future work

As stated in Christopher's blog post there are additional data sources in the iOS notifications directory. I plan on making parsers for these as well. Like everyone else on this floating rock in space, when can have to many things but the thing we will never have enough of is time. If only the days had more hours and our bodies less need for sleep.


As always I can be reached on twitter @AlexisBrignoni and email 4n6[at]abrignoni[dot]com.


Saturday, August 10, 2019

ARTEMIS - Android support for APOLLO

Short Version

Introducing a Python 3 script, with corresponding modules, that extend Sarah Edwards' APOLLO framework support to Android devices.

ARTEMIS

Name:
ARTEMIS (Android Review Timeline Events Modular Integrated Solution)
Purpose:
  1. Python 3 script that parses Android UsageStats XML files for automatic ingestion by APOLLO. 
  2. Add pattern of life analysis in Android devices to APOLLO.
  3. Continue to evolve ARTEMIS so it can parse additional non APOLLO supported data sources for APOLLO ingestion and analysis.
New APOLLO modules for Android devices:
  1.  UsageStats 
  2.  ContextLog 
  3. SamsungMembersDeviceEvents 
  4. SamsungSmartManagerUsageLog 
Update:
Converted the original Python 2 APOLLO script to Python 3. Works on Windows 10.
Pre-requisites:
 Full file system extraction of Android device will produce best results.
Location:
ARTEMIS, updated APOLLO, and Android APOLLO modules can be found in the following fork/branch: https://github.com/abrignoni/APOLLO/tree/py3-branch here: https://github.com/mac4n6/APOLLO
Long Version 

When Sarah Edwards released APOLLO on November last year I was highly impressed by the importance of aggregating specific pattern of life data (PLD) from iOS devices in a timeline format. Since then I have been looking for PLD in Android devices that can provide the same type of insight that APOLLO aggregates for iOS devices. This search motivated me to create the DFIR SQL Query Repo, a UsageStat XML parser, and multiple Magnet Forensic Artifact Exchange custom artifacts. Yesterday as I was driving home from work I thought about how to aggregate all those PLD artifacts in an APOLLO like format. Then it hit me, not a car but the idea of feeding non-sqlite Android files to APOLLO for parsing and analysis. Hence ARTEMIS was born.

Why

If you are not familiar with APOLLO you will be well served to do so. It can be found here. The Python 2 script has been instrumental in many DFIR cases. In order to extend the functionality I had to port it to Python 3. The only issue with APOLLO was that it was coded in Python 2 which will reach end of life by the end of 2019. For reasons unknown to me I couldn't make APOLLO work in a Windows environment even if I had Python 2 installed. I depended on my trusty Apple computer but that required moving data to and from my main DFIR boxes. By coding APOLLO in Python 3 my Windows machines can play nicely with it. Also since I code in Python 3 I could integrated with my nascent ARTEMIS idea.

How

First order of business was to convert APOLLO to Python 3. Thanks to the Python Modernize project it was trivial to do so. With APOLLO properly converted the next step was to find a way to prepare the largest PLD source in Android for APOLLO ingestion.

One of the main data sources APOLLO leverages for PLD in iOS is the SQLite knowledgeC database. There is no analogous SQLite database in Android. Thanks to Jessica Hyde's research I was made aware that similar PLD information is contained in the UsageStats XML. In order to have APOLLO ingest UsageStats data it had to be converted from XML to SQLite. ARTEMIS purpose is to do this conversion.

Last step was to create a module so APOLLO could parse this new SQLite database. I created a module for that purpose. Also in order to avoid having to call to scripts and move things around manually I use most of the same arguments APOLLO uses so I can have ARTEMIS hand off processing to APOLLO without user intervention assuming both scripts are located in the same directory.

Release

The following images will provide a visual on the results the user can expect ftom the APOLLO/ARTEMIS workflow.

  1.  Have APOLLO and ARTEMIS in the same directory
  2. ARTEMIS will use the YOLO option in APOLLO as default since APOLLO lacks any Android specific arguments. 
  3. Place your Android data sources aggregate in a directory. ARTEMIS, like APOLLO, will not parse forensic images. It will search logical files only.
  4. Place APOLLO modules that support your Android specific SQLite databases. At a minimum include the UsageStats module.
  5. Run artemis.py. After done it will call apollo.py to finish processing.
The following image shows both scripts in the same directory. Your Android logical files will be located in the Data directory. The Android APOLLO modules are in the Artemis_Modules directory. The modules in the Artemis_Modules directory are a product of the research presented at the SANS DFIR Summit 2019. For details on the data these modules parse see here.


Both Python 3 scripts in the same directory 
ARTEMIS uses almost all of the same arguments as APOLLO.

Arguments

Execute the following command:
python artemis.py -o sql Artemis_Modules Data
UsageStats being processed and turned into a SQLite DB
 When ARTEMIS is done with the UsageStats it will call APOLLO to parse the data stores.

Yolo option selected by ARTEMIS
As seen at the end of the previous image the timeline will be found in the apollo.db file. I used the following SQL statement:

SELECT
datetime(Key/1000, 'unixepoch', 'localtime') as time,
Activity,
Output,
Database,
Module
FROM APOLLO order by time asc

End product
It is awesome to see how multiple PLD sources come together to paint a more rich picture of what activity transpired on a device at a particular time.

Future development

A big part of ARTEMIS will be to add support for additional Android PLD sources that are not in SQLite form. In the immediate future I will work on having ARTEMIS automate the conversion of iOS mobile installation log data into a SQLite format for APOLLO ingestion.

Obligatory WARNING!!!!!

The output of these scripts is for lead purposes only. Verification by you is not optional. Be aware that there is always danger involved with timestamps from multiple disparate sources. Always verify the provenance of the data and how the timestamps related to each other.

As always I can be reached on Twitter @AlexisBrignoni and email 4n6[at]abrignoni[dot]com.