EXIF Capturing and Forensics

first and foremost, today is the Persian ancient new year (Nowruz), the first day of spring. Happy new year to all Persians.

but let me get back to our post title, oh Exif, a simple data structure which can reveal so many great information to intruders or investigators. needless to say, EXIF ( the exchangeable image file format ) normally contains some key information regarding the camera model, the place of taken picture and other precious data. with the advent of social medias and sharing the unlimited number of pictures on the internet, this is a good idea to investigate through countless of pictures to see What else can be taken except the content of the image. as our previous post's code was written in python, I follow the same programming language.  you need to download and install Python Image library first at this URL: http://www.pythonware.com/products/pil/

Then you need to take to the second step, test whether your library is installed properly or not.

run this code in python interactive terminal or save it into a file and run it.

from PIL.EXIFTags import GPSTAGS
gps ¼ GPSTAGS.items()
gps.sort()
print gps

taking a deeper look at what we got from the result is a dictionary of GPSTAGS. some of the key important items are GPSLONGITUDEREF and GPSLATITUDEREF which we need to take a look at it later on. Then, we need to know more about GPSTAGS, here is a brief introduction of what they are. the text below is quoted from Python Forensics Book: 

"GPSLatitudeRef: Latitude is specified by either the North or South position in degrees from the equator. Those points that are north of the equator range from 0 to 90 degrees. Those points that are south of the equator range from 0 to -90 degrees. Thus this reference value is used to define where the latitude is in relationship to the equator. Since the value latitude is specified in EXIF as a positive integer, if the Latitude Reference is “S” or South, the integer value must be converted to a

negative value.

GPSLatitude: Indicates the latitude measured in degrees. The EXIF data specify these as whole degrees, followed by minutes and then followed by seconds. GPSLongitudeRef: Longitude is specified by either the east or west position in degrees. The convention is based on the Prime Meridian which runs through the Royal Observatory in Greenwich, England, which is defined as longitude zero degrees. Those points that are west of the Prime Meridian range from 0 to -180 degrees. Those points that are east of the Prime Meridian range from 0 to 180 degrees. Thus this reference value is used to define where the longitude value is in relationship to Greenwich. Since the value of longitude is specified in EXIF as a positive integer, if the Longitude Reference is west or “W,” the integer value must be converted to a negative value.

GPSLongitude: Indicates the longitude measured in degrees. The EXIF data specify these as whole degrees, followed by minutes and then followed by seconds. "

for making large advances, always you need to follow other works and then make some modification into it and avoid reinventing the wheel. the source code below is intended to scan a path and gives you some information on which picture contains EXIF data.  when the for loop iterates, each of those pictures will be sent to a function of a class named _modEXIF which is responsible for the calculation of longitude and latitude and to see is there any EXIF tag or not to proceed the rest.

import os
import _modEXIF

TS = 0
MAKE = 1
MODEL = 2

# define a directory to scan
scanDir = "C:\\pics\\"
try:
        picts = os.listdir(scanDir)
except:
        print "error invalid directory"
        exit(0)
for aFile in picts:
        targetFile = scanDir + aFile
        if os.path.isfile(targetFile):
                gpsDictionary, EXIFList = _modEXIF.ExtractGPSDictionary(targetFile)
                if (gpsDictionary):
                        # Converted to degrees
                        dCoor = _modEXIF.ExtractLatLon(gpsDictionary)
                        lat = dCoor.get("Lat")
                        latRef = dCoor.get("LatRef")
                        lon = dCoor.get("Lon")
                        lonRef = dCoor.get("LonRef")
                        
                        if ( lat and lon and latRef and lonRef):
                                print targetFile + ":" + EXIFList[TS] + ":"  + EXIFList[MAKE]+ ":" +
                                EXIFList[MODEL]+ ":" +  str(lat)+','+str(lon)
                        else:
                                print "WARNING", "no GPS EXIF Data for "
                else:
                        print "no dictionary found in " + targetFile

and the class goes like this

import os # Standard Library OS functions
from PIL import Image
from PIL.ExifTags import TAGS, GPSTAGS
import sys
def ExtractGPSDictionary(fileName):
        try:
                pilImage = Image.open(fileName)
                EXIFData = pilImage._getexif()
        except Exception:
                print sys.exc_info()[0]
                return None, None
        imageTimeStamp = "NA"
        CameraModel = "NA"
        CameraMake = "NA"
        gpsDictionary = {}
        if EXIFData:
                for tag, theValue in EXIFData.items():
                           tagValue = TAGS.get(tag, tag)
                           if tagValue =='DateTimeOriginal':
                                imageTimeStamp = EXIFData.get(tag)
                           if tagValue == "Make":
                                CameraMake = EXIFData.get(tag)
                           if tagValue =='Model':
                                CameraModel = EXIFData.get(tag)
                        # check the tag for GPS
                           if tagValue == "GPSInfo":
                                # Found it !
                                # Now create a Dictionary to hold the GPS Data
                                for curTag in theValue:
                                        gpsTag = GPSTAGS.get(curTag, curTag)
                                        gpsDictionary[gpsTag] = theValue[curTag]
                basicEXIFData = [imageTimeStamp, CameraMake,CameraModel]
                return gpsDictionary, basicEXIFData

        else:
                return None, None

def ExtractLatLon(gps):
        # to perform the calculation at least lat, lon, latRef and lonRef are needed
        if (gps.has_key("GPSLatitude") and gps.has_key("GPSLongitude")
        and gps.has_key("GPSLatitudeRef") and gps.has_key
        ("GPSLatitudeRef")):
                latitude = gps["GPSLatitude"]
                latitudeRef = gps["GPSLatitudeRef"]
                longitude = gps["GPSLongitude"]
                longitudeRef = gps["GPSLongitudeRef"]
                lat = ConvertToDegrees(latitude)
                lon = ConvertToDegrees(longitude)
                if latitudeRef == "S":
                        lat = 0 - lat
                # Check Longitude Reference
                # If West of the Prime Meridian in
                # Greenwich then the Longitude value is negative
                if longitudeRef == "W":
                        lon = 0- lon
                gpsCoor = {"Lat": lat, "LatRef":latitudeRef, "Lon": lon,
                "LonRef": longitudeRef}
                return gpsCoor
        else:
                return None

def ConvertToDegrees(gpsCoordinate):
        d0 = gpsCoordinate[0][0]
        d1 = gpsCoordinate[0][1]
        try:
                degrees = float(d0) / float(d1)
        except:
                degrees = 0.0
        m0 = gpsCoordinate[1][0]
        m1 = gpsCoordinate[1][1]
        try:
                minutes = float(m0) / float(m1)
        except:
                minutes=0.0
        s0 = gpsCoordinate[2][0]
        s1 = gpsCoordinate[2][1]
        try:
                seconds = float(s0) / float(s1)
        except:
                seconds = 0.0
        floatCoordinate = float (degrees + (minutes / 60.0)+(seconds /
3600.0))
        return floatCoordinate

to sum up, you can check out the below link to find out which location those numbers refer to.
http://www.latlong.net/Show-Latitude-Longitude.html

Network Forensics - python scripting

as a practice and establishing a baseline in our blog posts, it is better to start off with python scripting. regarding the title of this post, numerous articles are on the internet and so many great books have been written in python security. the idea behind such an approach ( python scripting ) is to cover up the deficiencies of some custom or creative tools on security field. whenever you want to think about network forensics, you should consider about sockets, the fundamental building block which allows interacting with other devices. The socket API which most operation systems use today is based on Berkeley sockets. Berkeley sockets were used for the first time with UNIX BSD version in 1993. later, they released a free version that is used in nowadays use across different operation system.
for more information related to Berkeley socket, you can check out Wikipedia page.
for instance, if you have ever tried to write a client/server TCP application, the following flowchart must be observed by the programmer.

some methods or functions which commonly use in Socket programming by many today programming languages are presented in the above illustration.  you can find more information about how these methods and how to build network programs in python or any other languages on network programming books.

if I want to write about socket programming in python, it may more cumbersome for readers to read about it in a single post and I would prefer to write a simple code which does some sneaky information gathering through network access. this program is developed to collect information about open ports on a local or remote machine.  as you know for sure, many well-known ports are used by famous services and programs and that's why we need to specify them in an array. each try may take several seconds or minutes and we set a timeout for connection in socket object. in this code, two functions are responsible for getting the work done. first is to call check_port and second if the result indicates the port is open, then check_port will check what that port is for. this is a sample of running the code.

for more investigation, you can manipulate the code in any way or make it parallel for speeding it up

 
#!/usr/bin/python

# developed by Mansoor (manz@digitz.org)
# insecurebytes.blogspot.com

import socket,sys,time,datetime,argparse,os

line = "+" * 80


ip = socket.gethostbyname("127.0.0.1")


openports = []
common_ports = {

        '21': 'FTP',
        '22': 'SSH',
        '23': 'TELNET',
        '25': 'SMTP',
        '53': 'DNS',
        '69': 'TFTP',
        '80': 'HTTP',
        '109': 'POP2',
        '110': 'POP3',
        '123': 'NTP',
        '137': 'NETBIOS-NS',
        '138': 'NETBIOS-DGM',
        '139': 'NETBIOS-SSN',
        '143': 'IMAP',
        '156': 'SQL-SERVER',
        '389': 'LDAP',
        '443': 'HTTPS',
        '546': 'DHCP-CLIENT',
        '547': 'DHCP-SERVER',
        '995': 'POP3-SSL',
        '993': 'IMAP-SSL',
        '2086': 'WHM/CPANEL',
        '2087': 'WHM/CPANEL',
        '2082': 'CPANEL',
        '2083': 'CPANEL',
        '3306': 'MYSQL',
        '8443': 'PLESK',
        '10000': 'VIRTUALMIN/WEBMIN',
        '135' : 'unknown'
}

starting_time = time.time()
print "+" * 40
print "\tSimple Port Scanner..!!!"
print "+" * 40


print "Scanning started at %s" %(time.strftime("%I:%M:%S %p"))

def check_port(host, port, result = 1):
        try:
                sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                sock.settimeout(0.5)
                r = sock.connect_ex(("127.0.0.1", port))        
                if r == 0:
                        result = r
                sock.close()
        except Exception, e:
                pass

        return result

def get_service(port):
        port = str(port)
        if port in common_ports: 
                return common_ports[port]
        else:
                return 0


try:
        print "Scan in progress.."
        print "Connecting to Port: ",


        for p in sorted(common_ports):
                p = int(p)
                print p,        
                response = check_port("127.0.0.1", p)
                if response == 0:
                        openports.append(p)

        print "\nScanning completed at %s" %(time.strftime("%I:%M:%S %p"))
        ending_time = time.time()
        total_time = ending_time - starting_time
        print "=" * 40
        print "\tScan Report: 127.0.0.1"
        print "=" * 40
        
        total_time = str(round(total_time, 2))
        print "Scan Took %s seconds" %(total_time)

                
        if openports:
                print "Open Ports: "
                for i in sorted(openports):
                        service = get_service(i)
                        if not service:
                                service = "Unknown service"
                        print "\t%s %s: Open" % (i, service)
        else:
                print "Sorry, No open ports found"

except KeyboardInterrupt:            
        sys.exit(1)

A Survey of Reverse Engineering Tools for windows platform - part two

in addition to my previous post, let's add more info regarding reverse engineering tools. so far you have read about what the categories of reverse engineering tools are. before digging into the heart of the matter and introduce some tools of each section, let's take a look at a comprehensive reverse engineering process.

you can comprehend how the flow goes from the beginning step ( having an executable ).

hex editors:

hex editors as we mentioned earlier, give you this ability to make some changes to executables and DLL files. they mainly come handy when you need to change a specific piece of instruction or a string. for instance, you might want to change messageBox call to No-OP ( no operation )  - more info. for understanding these functions or System API such as messageBox if you wonder how you can find them, you need to check some references out to get some information about these APIs. most of these functions are located in Native DLLs such as kernel32.dll or USER32.dll. 

an advanced hex editor which is worthy to know is WinHex, as you can see in the following figure, hexadecimal and ASCII equivalents of a sample binary program are displayed.

however, another Hex editor named Hex workshop is out there which is capable of doing some mathematical operations. the aforementioned editor is able to interpret little-endian(intel x86)  and big-endian (power PC) and has some features such as character distribution and data visualizer as well.

Debugger / Disassembler:

many believe that most two well-known programs in this category are OllyDbg and IDA pro. OllyDbg is an application-level debugger which is used mainly on different versions of windows operation systems.  it is hard to mention all of its features but some of the most frequent ones are disassembly section, hex dump, stack window and the lovely CPU register window. OllyDbg supports for setting conditional breakpoint, memory dumping and bypassing the IsDebuggerPresent API. on the other side, IDA Pro provides accurate analyzes, disassembling of the binary program in color-coded. well-organized and easy-to-read way and like Ollydbg, it can give you some common options such as import and export viewing, hex editting, string extraction.

even so, they are some other debuggers or disassemblers which can be found as a good choice sometimes such as SoftIce, TRW or some language-specific debuggers like DeDe, VBReFormer.

Decompiler:

decompilers are so fancy because they open the doors of fortunate to you in a second without any efforts. as a first example, JAD is a java decompiler which is written in C++ but it has its own drawbacks while decompilation process. the second example is JODE which sometimes it performs better than JAD on java files. we just covered some decompilers related to JAVA language but you can find many on the internet regarding other languages to help you to get the main source code if everything is not meant to stop you.

source code obfuscators:

when you send your source-code to an obfuscator, mostly obfuscator gives back an obfuscated output of your file. clearly, you should do this when you want to give a compiled version of your binary file to a customer. in a simple obfuscation process, they would change variable and function names to something very short or meaningless. you previous indentation and space will be removed to make your code less readable in case of decompilation. as an example, PreEmptive solutions offers obfuscator for both dot.NET and Java languages.

some binary obfuscators which frequently used encryption and packing, they add a decryption routine to the executable to achieve decryption at run time. at the moment, the pure binary is visible in the memory and this is a good chance to manipulate the binary on the fly or clone it somewhere on your hard drive.  as an example, ASProtect uses a technique called stolen bytes. Stolen bytes refers to an anti-memory dumping technique that requires the deletion of a section of code quickly after it is executed

                           

A Survey of Reverse Engineering Tools for windows platform - part one

interestingly, reverse engineering will captivate you after learning tips and tricks of it. as my first post regarding reverse engineering, it is better to explain what reverse engineering is. in fact, reverse engineering consists of several steps or processes to extract knowledge or comprehend the design flow of a human-made product. this product can be anything in this world. from industrial machines to tiny software such as a MP3 converter program. designers or it is better to be said, programmers ( not a significant number of them ) in our context, do their best to place some preventive mechanism to stop crackers. on the other hand, virus writers would obfuscate their runtime behaviour to deceive reverse engineer analysists while examination or to slip sneakily under the radar of anti-viruses. reverse engineer in software context defined by Chikosfky. as they published a research paper, they state: "Reverse engineering is the process of analysing a subject system to create representations of the system at a higher level of abstraction." 

obfuscation is a famous terminology in software design and reverse engineering which is referred many times by books, articles and so on. in essence, obfuscation or anti-tempering techniques would protect and guard your program against those ones whose intent is to modify your application or somehow re-engineer it. 

additionally, there are many fruitful tools which can be handy in different circumstances. to understand the behaviour of a specific program, you need to choose a debugger or disassembler at first to identify or gain the basic knowledge of it. once you figure out the program is obfuscated or dimmed, the next step must be the identification of obfuscator whether it is a ready-made one or a custom personal one is. some PE editors would recognise and do the responsibility.  to restore everything to a normal state, an unpacker must be used and in those odd cases when you confront with a custom personal scheme, you should do the process of unpacking manually while this requires high skill. 

after defeating the obfuscation phase, a debugger goes for the rest of way. it is important to realise that by using a debugger or disassembler, you can change the Strings, Trace some outgoing API calls and input/output functionality. typically, you would select your debuggers or disassembler based on the programming language of that executable. therefore, if you have a program which is written in VB6 and another one is written in C#, evidently, two different disassemblers would be used in this situation.

each one presents some unique features to ease the process of disassembling. you may often see that program is resistant to making any changes and within itself, some preventive mechanisms such as debugger detection, checksums add some trouble and hardship. 

categories of Reverse engineering tools

after all, an arsenal of tools has been developed over the recent years and in the following four main categories, most of them can be included.

Hex editors: they provide a representational of base-16 or hexadecimal format of the binary data. they will also show the equivalent of base-16 in different ways, for instance: ASCII or Unicode 

Disassembler/Debugger: these programs translate base-16 into assembly language while debuggers would run a program and give the user facility of terminating or pausing the application at a certain point. in most cases, these two kinds of software would be in a single product.

Decompilers: these programs would translate the executable program directly into source code.

Related tools: tiny programs which play an important role such as obfuscators, PE editors, memory analyser and dumper, packers and unpackers