Local Storage - Firefox Focus Privacy Browser Artifacts in Android

Short version: 

Data placed by websites in the Local Storage folder LevelDB database for the Firefox Focus Privacy Browser (FF-Focus) app will remain and be accessible even after browsing history deletion.  The data will be located at '/userdata/data/org.mozilla.focus/app_webview/Local Storage/leveldb' directory.

Long version: 

First I want to thank Ryan Benson (@_Ryan Benson) and Phill Moore (@phillmoore) for their help and guidance with this analysis.  I am thankful for their generosity in regards to their expertise and time. Any and all shortcomings with this analysis obviously reside with me. And like anything else in life always make sure to validate, validate, validate.

FF-Focus is a privacy oriented browser developed by the Mozilla organization. As stated in the picture below the app blocks ads, trackers and deletes all browsing related items like history and cookies.

No ads, no history, no nothing!!!

Testing 

I wondered how much data could remain, if any, after the app completed its privacy routines. In order to test I used some of the equipment mentioned in the following link for testing and analysis.

Android - Mobile testing hardware and software

After downloading the app to my exemplar phone, I did some browsing and imaged the device before closing the app. As can be seen below, the app had created all the files we would expect to see in a typical browser directory when in use. The cache folder has contents and the current session cookies are accessible.

Cookies are there. Yum.

If called to image a target device, being able to do so when FF-Focus is open would be the ideal thing, it will also be the most unlikely scenario. After the app is closed all those browser artifacts are gone.

All gone except...

The was only one remaining directory, Local Storage, and within it the LevelDB folder. The files within it were:

• 000003.log
• CURRENT
• LOCK
• LOG
• LOG.old
• MANIFEST-000001

After a quick view of contents of each file, the 000003.log seemed immediately relevant because it contained the domain names of the websites I had browsed previously.

I can see domain names.

After consulting with Phill Moore he referred me to Ryan Benson and his excellent presentation on browser forensics which he delivered at the SANS DFIR Summit 2017. The presentation can be seen at Youtube in the following link.

Deciphering Browser Hierogplyphics - min 17:20 onwards

I strongly encourage the reader to check out the whole of Ryan's presentation since my summary of the aforementioned data stores will do no justice to the depth of the explanation found in the video. 

In summary, the files seen in the Local Storage directory are responsive to website data stores that use the Google developed LevelDB database. This data store is based on key - value pairs unlike the traditional SQLite databases found everywhere in mobile devices. If a web page inserts a key-value pair where either the key or the value is the domain website, it will remain there even after the app deletes all the browsing history. It is also interesting to note that new key-values are added to the data store across browsing sessions.

These observations led to the following questions:

• Do all web pages leave data in the data store?
• Can all the domain names in the data store be attributable to a deliberate user request for a page?
• How are these records accessed? How are they ordered? Are there any timestamps available? Any ordering values?

Local Storage usage by web sites

In order to have an idea of what type of values a web page might place in the data store I examined the values being pushed to the browser by using the developer options functionality in Chrome and Firefox on my testing desktop. Ryan's video has a detailed explanation and a visual of such tools. 

The next image shows one of the pages I visited using FF-Focus for testing.

There is no key-value for the domain name.

Notice there is no key-value pair for the domain name. As expected I did not find the domain name for the visited page in the data store of the phone either. This seems to indicate that not all websites will leave a trace of the domain name in the store in question.

Now lets look a the following image from CNN.com

Local Storage values from CNN.com

The values in the image mirror the values found in the data store in the FF-Focus app directory. And yes, the domain name was there. But also notice that some domain names I did not visit explicitly, like Youtube, do appear in the data store. It is my hypothesis that video content played within the page will leave such traces in the data store. This needs further testing.

It is also worth nothing that these behaviors in FF-Focus might also be mirrored in most if not all browsers. 

Data Extraction

In order to view the key-value pairs more easily I needed to dump the data from the data store. The data was located in the 000003.log file. For this purpose Ryan shared with me a python script he quickly wrote that uses the leveldb wrappers to access data store. For my testing I collected 8 versions of the data store at different times after browsing a growing number of web pages per extraction. The following process was the most consistent way of dumping the values in the data store. It worked for all extractions tested.

1. Export the 'leveldb' directory from Local Storage and run the lvl.py script here. The script will access the data in the log file and move it over to a .ldb file. This is expected behavior of a LevelDB data store. I used the Linux version of the SIFT Workstation due to limitations with LevelDB python support in Windows.
   
   The usage is as follows:
   Run 'python lvl.py ~/location-of-the-leveldb-folder/leveldb'
   
   The script will dump the contents to the file to the screen (or to a file using a redirector) as it iterates through it. In some of my test data the contents did not show. The only consistent way of accessing the data was by dumping the data directly from the generated .ldb file.
   
2. To dump the data from the generated .ldb file I used the ldbdump tool found in Golang. Steps for my testing environment:
   
   - Install Golang for Windows. Link here.
   - Install Git for Windows. Link here.
   - Get the Golang leveldb repo. Run 'go get github.com/golang/leveldb'.
   - Navigate to 'C:\Users\username\go\src\github.com\golang\leveldb\cmd\ldbdump'.
   - Run 'go build -o ldbdump main.go'.
   - Take the ldbdump file and add the .exe extension.
   - Run 'ldbdump target.ldb > exportedldb.txt' where target.ldb is the extracted file name.

The exported data will look something similar to this:

Key value pairs. Some hex.

Pending items

After the data is extracted a noticeable pattern of hex values accompany the key in the data store. Currently I don't know what those values mean. They could be artifacts from the browser, some sort of data store ordering scheme or something totally different.

The formatting of the exported data still needs work. Further work on a script that can address the need is also to be done.

Why does this matter?

Being able to have some clues regarding subject browsing habits might be key to an investigation. This is even more essential when the involved individual/s are using a no history, privacy centric application like FF-Focus.

Android - Mobile testing hardware and software

Android Hardware

• Exemplar phone 0: Nexus 6P - Rooted - OS 8.1.0 
• Exemplar phone 1: Samsung Galaxy Prime - Rooted - OS 5.1.1
• Exemplar phone 2: Samsung S3 - Rooted - OS 4.1.2

Imaging Software

• Magnet Acquire 2.2.6
• Cellebrite UFED4PC

Image file system viewer / exporter /processing

• FTK Imager 3.4.2.2
• Cellebrite Physical Analyzer
• Autopsy / Sleuth Kit

Database viewer

• SQLite Browser

Scripting/Developer software

• Python
• Golang

OS platforms

• Windows 10
• SIFT Workstation - Linux

Was the video played? - Android video player apps

Was a particular video in an Android device viewed? At what time? How many times? Was the video not played at all? Where would such information reside within the Android file system? Since there are so many video player apps for Android this blog post will do a quick rundown on played video artifacts for five of the most popular players per the Google Play Store.

For this analysis the following tools were used:

• Exemplar phone - Samsung Galaxy Prime phone - rooted - OS version 5.1.1
• Magnet Acquire 2.2.6 - Full/Physical extraction.
• FTK Imager 3.4.2.2 - View file system and export files.
• SQLite Browser - View database contents.

Methodology:

1. Record three videos with the exemplar's camera functionality.
2. Download five popular video players from Google Play Store.
3. Play two out of three videos with each player.
4. Use Magnet Acquire to get a physical image of the exemplar's storage.
5. Use FTK Imager to locate each applications' files/databases and export them for review.
6. Use SQLite Browser to view the content of the exported files if needed.

All paths are located in the userdata folder,  identified in the rest of the blog post as [root].

VLC

One of the most popular video players with 100+ million downloads. 

Pertinent file location and name: [root]/data/org.videolan.vlc/app_db/vlc_media.db

The table named 'media' within the vlc_media.db database contains most of the information we are looking for.

Schema for the 'media' table.

Notice the 'play_count', 'filename', and 'duration' fields. As expected the video that was not played had a null value on the 'play_count' field as opposed to the value 1 for the played videos. The table named 'file' contains the location/storage of the played videos.

MX PLAYER

Another popular video player with 100+ million downloads.

Pertinent file location and name: [root]/data/com.mxtech.videoplayer.ad/databases/medias.db

The table named 'VideoFile' within the medias.db database has the following schema:

Schema for the 'VideoFile' table.

This table has more detail than the one found in VLC. Notice the 'LastWatchTime' and the 'FinishTime' fields. The two played videos had a Unix timestamp (see here for time conversion) in the 'LastWatchTime' field. One of the played videos I did not play all the way to the end. That one had no timestamp in the 'FinishTime' field. The video that was not played at all had null values on both fields. Another table named 'VideoStates' contained additional information like the location of the files but only for the files that were played.

ARCHOS

Unlike the previous apps, Archos does not identify native videos automatically. The user has to navigate to the folder where the videos are located in order to play them.

Pertinent file location:[root]/data/com.archos.mediacenter.videofree/databases/media.db

The table named 'files' within the media.db database has the following schema:

Schema for the 'files' table.

The table contained not only data about the video files but also contained data about files that are not playable media. At first impression it seemed to populate the table with the items it encountered as the user navigated to the desired video files location. More testing needs to be done to confirm this. In addition the videos, after played, did not create entries in fields like 'Archos_lastTimePlayed' where one would expect to encounter timestamps. Again, more testing is in order for this app.

PLEX

It is of notice that on the Google Play Store page for Plex it says that the app has a tendency to crash if selecting local and synced content.

Yup.

I found out that the warning was well warranted. None of the local videos would play since it crashed the app every single time. A log detailing the app's attempt at playing the videos was located at [root]/data/com.plexapp.android/Plex Media Server/Logs/Plex Media Server.5.log

Fail log.

No SQLite database was found containing additional information. Further testing with files that are neither local nor synced will need to be done to ascertain if a database is used to store any data about the played files.

LOCALCAST

This app is not a video player but a casting application for Chromecast and other Google devices. I selected two videos to play on my Google home. They did.

Pertinent file location:[root]/data/de.stefanpledl.localcast/databases/notes-db

The table named 'RECENT_ITEM' within the notes-db database has the following schema:

Schema for 'RECENT_ITEMS' table.

The table contained two entries, one for each video I had sent to my Google Home for playback. Not much historical data contained in this table or any other I could find. The table named 'QUEUE_ITEM' contained and entry for each of the three videos. Again no additional historical information was located.

As time permits I will try and fill in more details regarding the apps described here as well as look into new ones as they become popular. As always, for case work, one must do multiple rounds of testing and validation before arriving at a conclusion. Everything stated previously is just a guide for further testing.

Finding Discord chats in OS X

After much searching the Discord cache folder is located here:

/Users/myusername/library/Application Support/discord

The cache folder follows the same file structure as the one found in Windows.

Discord cache folder in OS X

The following links will explain how to extract the json chat objects and how to convert them to html or xls files using a Python script. Although written originally for objects found in Windows the extraction and conversions steps apply all the same.

Extraction of json objects:

 https://abrignoni.blogspot.com/2018/03/finding-discord-app-chats-in-windows.html

Discord json chats conversion to html or xls.

https://github.com/abrignoni/Discord-JSON-chat-conversions

As background on how I located the correct folder I took the following steps:

1. Created a virtual OS X using Virtual Box. Virtual storage was VDMK format.
2. Installed the Discord program.
3. Logged in to my test account. The chats were synchronized from the ones in Discord servers.
4. Turned off the virtual machine. Created a snapshot.
5. Tried to process the snapshot with Autopsy. Wouldn't take it.
6. Made a clone of the machine in order to consolidate the snapshot and the image into one. Autopsy still wouldn't take it.
7. Installed Qemu. Converted the VMDK to RAW using the following command:
   
   quemu-img.exe convert -f vmdk 'J:\my-clone-mac-disk.vmdk' -O raw my-mac-disk.raw
8. Processed the raw file with Autopsy. In the keyword search section I added some of my test chats content and some other obvious terms like 'discord'.
9. Looked at the keyword search results. These lead me to the proper folder location described above.

At the end of the day I not only found what I was looking for but also learned about OS X folder structures and how to manipulate virtual machines. Everything you do is an opportunity to learn and share with others.

How to convert UNIX Epoch timestamps in SQLite DB fields to local time.

Short answer:
 select mychatfield, datetime(mytimefield/1000, 'unixepoch', 'localtime') from mytable

Detailed answer:
Most SQLite databases used in Android applications store their time stamps in UNIX time, also know as UNIX Epoch. Unix time defines a point in time as "the number of seconds that have elapsed since 00:00:00 Coordinated Universasl Time (UTC), Thursday, 1 January 1970, minus the number of leap seconds that have taken place since then." It goes without saying that these date are not stored in a way that makes them understandable to our consumers.

If your forensic tool does not parse a particular SQLite DB content automatically here is a quick way of converting those UNIX time stamps to local time.

For this example I will use an Tumblr Android SQLite DB that was extracted using Magnet Forensics Acquire and FTK Imager. To view the contents I used DB Browser for SQLite. The path location of the extracted database in my exemplar phone was:

userdata (ExtX)/Root/data/com.tumblr/databases/Tumblr.sqlite

Here is how the table looks with some sample data using DB Browser for SQLite.

Notice the timestamp field. A long string of numbers. Also notice the text field. The user content we are looking for if we are interested in chat content, for example. In order to present the chats with the corresponding time stamps in a human readable way we can use a SQL query to make the conversion.

Here is the query and the results:

select text as messages, datetime(timestamp/1000, 'unixepoch', 'localtime') as dates from messaging_message 

The datetime function takes three arguments in this example. The time string, UNIX epoch and local time modifiers. The reason the timestamp field values are divided by 1000 is due to the UNIX time being stored in milliseconds when the datetime function expects the UNIX time to be in seconds. By dividing we change the milliseconds to seconds.

In order to make the final column headers more descriptive change them in the query by using 'myfielname as newname' as seen above.

As always when doing these types of conversions on a case run validation tests with known data in a replica of the environment you are analyzing.

Discord JSON chats to XLS

Just added a script to convert the JSON chats to XLS spreadsheets.

Hopefully will be able to add batch functionality for all files in a folder as well as output selection all in one script by the end of the week.

• https://github.com/abrignoni/Discord-JSON-chat-conversions

@alexisbrignoni

Update 0:

Batch functionality and output selection done.

Update 1:

Added error handling via screen and error.txt file for chats that are unable to be converted to XLS.

Finding Discord app chats in Windows.

Discord on the desktop
In previous posts I discussed some ways of recovering and presenting Discord app chats from Android devices. This post will discuss how to find Discord chats in Windows machines and provide a simple way to visualize them.

As way of background Discord is a chat application whose target audience is people who play video games. Wikipedia states that the Discord app has 87 million unique users. With such a large user base it is interesting to find that current commercial tools do not parse Discord chat artifacts directly yet.

Location and Extraction
Like many Windows applications, the user generated files and configurations reside in the apps folder. For example:

C:\Users\SampleUser\AppData\Roaming\discord

The Discord AppData folder has the following structure:

Cache folder highlighted

The user activity files are located in the Cache folder. At first glance the chat files we are looking for are not immediately apparent.

Where are the chats?

A look at the Cache folder contents might seem familiar. It is the same file format of the Google Chrome cache located at

C:\Users\SampleUser\AppData\Local\Google\Chrome\User Data\Default\Cache

My default browser is Chrome and the Discord app uses the same storage structure. For comparison here is my Chrome cache folder view.

Same file structure

Since the file structures are the same it seem clear that the content we are looking for had to reside within the Chrome cache like folder structures in the Discord cache folder. Thankfully there are many tools that allow us to parse those structures and extract files from them. For this analysis I used the folowing tool:

ChromeCacheView v1.77 - Cache viewer for Google Chrome Web browser Copyright (c) 2008 - 2018 Nir Sofer

The tool will by default parse the Chrome cache at the default folder location in your computer. Just hit stop on the upper left corner and point the tool to the Discord cache folder. After processing the contents of the folder the tool will show something similar to the following:

Discord cache folder contents

The tool will allow you to print out all the metadata on screen to an HTML file. Even more useful is the ability to extract the actual files from the cache by selecting an entry in the list and pressing F4 (or via the menu.)

To find the chat files look in the URL column for addresses that end with "messages?limit=50".

messages?limit=50

These are the files we will export from the cache and will contain the chat messages. It is of note that they not always end in 50. In some cases they can end in 100. Also note that the name of these files is usually 50 as stated in the URL variable but that is no always the case. Some chat names start with the words After or Before followed by some sort of numeric ID. Hence the best way to identify them is to go by the ending of the URL column.

By clicking on file entry in the list one can see the pertinent metadata. It states that the files we are exporting are JSON files.

Content Type: application/json

Since they are JSON files they can be viewed in any regular file viewer. After exporting here is how one looks using Wordpad.

Content key is key

The formatting is hard to the eyes but understandable. Notice the content key, the value is the user generated chat. Each user generated block starts with the attachment key and continues with keys for multiple time stamps, user IDs, the chat content and the like.

In order to make it a little easier to read the values, the following script takes the json file contents and presents them as a collection of html  tables.

A little better

The best way to look at these files is to use Chrome since other browsers do not know how to decode certain characters, like emoji.

Look!!! Emojis!!!

The script is really simple and can be found here:

• https://github.com/abrignoni/Discord-JSON-chat-conversions

Script uses the json2html module that does the actual heavy lifting. It can be found here:

• https://github.com/softvar/json2html-flask

Pending will be the capability for the script to parse a group of chats in a folder instead of one by one.

I can be reached via twitter @alexisbrignoni

PD:

One can log into a target Discord account without the username and password by installing the Discord application on our forensic computer and then copying over the Discord app data folders from the examined computer. The program will require internet access for this to work. It goes without saying that consent to search or a search warrant is needed before attempting this type of access. 

-Brigs

Organization of American States - 37th Cybercrime Regional Workshop

Beautiful Guatemala City. 

What a great experience training and exchanging views with prosecutors from Costa Rica, the Dominican Republic, El Salvador, Guatemala, Honduras, Mexico and Panama in REMJA’s Cybercrime Regional Workshop.

Hope to be back soon...

Antigua Guatemala.
Worth visiting. Great local art and plenty of interesting history.

OEA flag and seals.

Welcoming ceremony. Event organized by the always
amazing @fiorella_mh

Lecture and case studies.

Cybercrime discussion with @TheJusticeDept prosecutors and
@OEA_Justicia subject matter experts.

@fiorella_mh

Honored to be able to speak on
mobile technology, social media, and dark markets

Dinner with faculty.

Viewing extracted Android app data using an emulator

In the previous blog posts I used free and open source forensic tools to view the content and file structure of the Android Discord app. After some testing and validation the presentation of the contents that the tools provide is understandable but not really intuitive nor user friendly.

What if you wanted to view the app content as the Android device presented it to the user without manipulating the device where the data originally came from? By using an Android emulator we can view the contents using the app itself as our viewer. Free tools will be used for this analysis.

Test Device

• Samsung SM-G530T Galaxy Grand Prime
• OS 5.1.1
• Rooted

 Tools

• Android Discord App with test data
• Magnet Forensics Acquire 2.0.3.8168
• Nox Player 5.0.0.1 - Android Emulator
• ADB - Android Debugging Bridge
• FTK Imager 3.4.2.6

Extraction

Using Acquire a physical source image of the phone is created.

Magnet Acquire Acquisition

With FTK imager the following folders are exported from the source image:

• /app/com.discord-1
• /data/com.discord

The app data folder contains the Discord application itself. It is named "base.apk".

Discord APK - base.apk

 The data folder has all the content related to the app. Be aware that different apps will require different folders to be identified and exported. For example some apps use the SD card to store data needed for the app to work properly. Using a test device, identify all the folders needed by the app then export them from the source image.

Discord Data Folder

After exporting the folders start the Nox emulator. Depending on the app you are working with make sure to change the settings on the emulator. By default the emulator starts in tablet mode.

Tablet Mode

If the app being worked on is from a phone, change the mode and resolution accordingly.

Change Mode & Resolution

Since I am using a test phone, here is how the emulator looks after the change.



Leave your emulator running since we will connect to it using ADB.

ADB - Connecting and pushing extracted data to the emulator 

We will use ADB to connect to the emulator and push into it the data repositories we identified previously. Nox comes with ADB already available in the main installation folder. For a default installation the Nox ADB will be found at C:\Program Files(x86)\bin\adb.exe or adb_nox.exe.

Open a command prompt at the previous path and connect to the emulator using the following command:

• adb connect 127.0.0.1:62001

To test connectivity use the following command:

• adb devices

If successful you should see the following on your screen.

Connected to Emulator

In another command prompt open a shell to the emulator using ADB.

• adb shell

ADB Shell

The first command prompt will be used to push/upload the apk. The second command prompt will enable us to interact with the emulator's file system.

Use the following command, in the first command prompt, to push the apk into the emulator into the corresponding directory:

• adb push base.apk /data/app

APK to Emulator

The application will appear on the home screen of the emulator. Press on it to initialize it. By starting the apk, the folder structures needed by the app will be created in the emulator.

After seeing the initial screen, press the Recent Taks button in the emulator (the square above the >> on the lower right side of the screen) and swipe away the app. 

Before we push the app data to the emulator we need to delete the folders that were created by the app when we initialized it.

On the second command prompt, the one with the ADB shell, navigate to the following folder:

• /data/data/com.discord

There you will see all the folders that were created by initialization. Use the following command to delete them:

• rm -r *

Delete App Folders

Go back to the first command prompt and push the extracted data app to the folder we just emptied.

• adb push com.discord /data/data/com.discord

Extracted App Data pushed to Emulator

If the push was successful the following will be seen on the prompt:

Successful Push

On your ADB shell prompt you will now see that the folders extracted from the image are now located within the emulator.

Successful Push - Folder Structure

To summarize so far we have:

1. Made a physical source image from the target device.
2. Extracted the target app and app folders (Discord.)
3. Push the extracted app to the emulator and initialize it.
4. Close the app and delete the app created folders.
5. Push the extracted app folders and their content to the emulator.

Go to the home page on the emulator and press the Discord app. All the content of the app can now be viewed in the emulator.

Discord Direct Message

Considerations

This method of review is really useful for showing the extracted data to non technical people. It is a great visual aid to see the data as the original user would have seen it herself. 

When doing this type of analysis we need to always adhere to best practices. For this example my workstation and emulator were connected to the internet. Notice the green dot next to the username at the bottom of the following image.

Logged In

Make sure your workstation is not connected to the internet since some applications will reach out and log in to the servers that the app uses. This could cause an issue where the analyst might inadvertently conduct a search/download of data from the server without proper legal authorization. 

Be aware that emulator analysis like this one might not work with applications that are closely integrated with the operating system, e.g. Google apps.

In my testing I have found that by closing the app or shutting down the emulator, the pushed data is no longer accessible and the app returns an invalid install error message. Make sure to take screenshots or video capture the pertinent screens before you close the app or shutdown the emulator. To access the app and data after shutdown will require reinstalling Nox and redoing the whole procedure again.

Discord App - Missing values not missing

* Oct 17, 2018 - Update with further insight here. *

Last post I did a quick overview of the Discord app for Android. At the time I commented that "both the messages and the usernames are missing the last letter as represented by the tools." Upon further review I figured out that
the alphabetic value of the last hex number at the end of every sentence can be decoded starting at E1h and ending at FAh.

For example the first 5 alphabetic values would be:

E1h = a
E2h = b
E3h = c
E4h = d
E5h = e

One has to only continue to map letters to hex values sequentially until the whole alphabet is represented.

By looking at the hex at the end of the sentence and comparing it to the corresponding alphabetic value one can figure out what is that last letter the tools used did not show us.

Go figure.