Nixyri/ghidra
1
0
Fork 0
mirror of https://github.com/NationalSecurityAgency/ghidra.git synced 2024-11-21 19:42:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/GT-2743_caheckman' into Ghidra_9.0.3

Browse Source
This commit is contained in:
ghidravore 2019-04-17 18:31:49 -04:00
commit 2172490738
8 changed files with 130 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
CONTRIBUTING.md
View File

@ -67,6 +67,31 @@ smooth process when submitting a patch:
Once available, please see the [developer's guide][devguide] for instructions to set up a suitable development environment.
## Timeline and Managing Expectations
As we continue to engage contributors and learn best practices for running a successful open source project, our processes
and guidance will likely evolve. We will try to communicate expectations as we are able and to always be responsive. We
hope that the community will share their suggestions for improving this engagement. Based on the level of initial interest
we receive and the availability of resources to evaluate contributions, we anticipate the following:
- We will initially prioritize pull requests that include small bug fixes and code that addresses potential vulnerabilities
as well as pull requests that include improvements for processor language specifications because these require a
reasonable amount of effort to evaluate and will help us exercise and revise our process for accepting contributions. In
other words, we are going to start small in order to work out the kinks first.
- We are committed to maintaining the integrity and security of our code base. In addition to the careful review the
maintainers will give to code contributions to make sure they do not introduce new bugs or vulnerabilities, we will be
trying to identify best practices to incorporate with our open source project so that contributors can have more control
over whether their contributions are accepted. These might include things like style guides and requirements for tests and
documentation to accompany some code contributions. As a result, it may take a long time for some contributions to be
accepted. This does not mean we are ignoring them.
- We are committed to integrating this GitHub project with our team's regular development work flow so that the open source
project remains dynamic and relevant. This may affect our responsiveness and ability to accept pull requests
quickly. This does not mean we are ignoring them.
- Not all innovative ideas need to be accepted as pull requests into this GitHub project to be valuable to the community.
There may be times when we recommend that you just share your code for some enhancement to Ghidra from your own
repository. As we identify and recognize extensions that are of general interest to the reverse engineering community, we
may seek to incorporate them with our baseline.
## Legal
Consistent with Section D.6. of the GitHub Terms of Service as of 2019, and Section 5. of the Apache License, Version 2.0, the project maintainer for this project accepts contributions using the inbound=outbound model.

129
DevGuide.md
View File

@ -1,33 +1,52 @@
# Developer's Guide
# Developer's Guide
## Catalog of Dependencies
The following is a list of dependencies, in no particular order.
This guide includes instructions for obtaining each of these at the relevant step(s).
This guide includes instructions for obtaining many of these at the relevant step(s).
You may not need all of these, depending on which portions you are building or developing.
* JDK 11 - We test and build using OpenJDK 11.0.2.
- https://jdk.java.net/11/
* Eclipse - It must support JDK 11. Eclipse 2018-12 or later should work. Other IDEs may work, but we have not tested them.
- https://www.eclipse.org/downloads/
* Gradle 5.0 - Later versions may work, but you'll need to modify our version check.
* A C/C++ compiler - We use Visual Studio 2017 on Windows, GCC on Linux, and Xcode (Clang) on macOS.
* Git - We use git-bash on Windows. Most Linux distros have git in their repos. Xcode provides git on macOS.
- https://gradle.org/next-steps/?version=5.0&format=bin
* A C/C++ compiler - We use GCC on Linux, Xcode (Clang) on macOS, and Visual Studio 2017 on Windows, .
- https://gcc.gnu.org/
- https://developer.apple.com/xcode/
- https://visualstudio.microsoft.com/downloads/
* Git - We use the official installer on Windows. Most Linux distros have git in their repos. Xcode provides git on macOS.
- https://git-scm.com/downloads
* Bash - This is moot on Linux and macOS. On Windows, we use MinGW. This may be distributed with Git for Windows.
* Bison and Flex - We use MinGW on Windows. Most Linux distros have these in their reposs. Xcode provides these for macOS.
* dex2jar
- https://osdn.net/projects/mingw/releases/
* Bison and Flex - We use win-flex-bison v2.5.17. These packages may also be available in MSYS (MinGW). Most Linux distros have these in their repos. Xcode provides these for macOS.
- https://sourceforge.net/projects/winflexbison/
* dex2jar. We use version 2.0.
- https://github.com/pxb1988/dex2jar/releases
* AXMLPrinter2
* HFS Explorer 0.21
* Yet Another Java Service Wrapper 12.12 - Only to build Ghidra package.
* Eclipse PDE - Only for the GhidraDev plugin.
* Eclipse CDT 8.6.0 - Only for the GhidraDev plugin
* PyDev 6.3.1 - Only for the GhidraDev plugin
- https://code.google.com/archive/p/android4me/downloads
* HFS Explorer. We use version 0.21.
- https://sourceforge.net/projects/catacombae/files/HFSExplorer/0.21/
- https://github.com/unsound/hfsexplorer/releases (newer versions)
* Yet Another Java Service Wrapper. We use version 12.12 - Only to build Ghidra package.
- https://sourceforge.net/projects/yajsw/files/yajsw/yajsw-stable-12.12/
* Eclipse PDE - Environment for developing the GhidraDev plugin.
- https://www.eclipse.org/pde/
* Eclipse CDT. We use version 8.6.0 - Build dependency for the GhidraDev plugin.
- https://www.eclipse.org/cdt/
* PyDev. We use version 6.3.1 - Build dependency for the GhidraDev plugin.
- https://sourceforge.net/projects/pydev/files/pydev/
There are many others automatically downloaded by Gradle from Maven Central and Bintray JCenter when building and/or setting up the development environment.
There are many, many others automatically downloaded by Gradle from Maven Central and Bintray JCenter when building and/or setting up the development environment.
If you need these offline, a reasonable course of action is to set up a development environment online, perhaps perform a build, and then scrape Gradle's cache.
## Install Development and Build Tools
If you're on Windows, install Git.
You may also need to install MinGW.
Many of the commands given below must be executed in Bash.
If you're on Windows, install Git, MinGW, Bison, and Flex.
Many of the commands given below must be executed in Bash (Use git-bash or MSYS from MinGW).
**IMPORTANT**: The bison and flex executables may be named `win-bison.exe` and `win-flex.exe`.
Our build cannot currently cope with that, so you should rename them to `bison.exe` and `flex.exe`.
Install OpenJDK 11 and make sure it's the default java.
@ -160,72 +179,28 @@ Some of Ghidra's components are built for the native platform.
We currently support Linux, macOS, and Windows 64-bit x86 systems.
Others should be possible, but we do not test on them.
#### decompile
Install bison and flex.
Ensure bison and flex are installed and in your `PATH`.
Now build using Gradle:
On Linux:
```bash
gradle decompileLinux64Executable
gradle buildNatives_linux64
```
On macOS:
```bash
gradle decompileOsx64Executable
gradle buildNatives_osx64
```
On Windows:
```cmd
gradle decompileWin64Executable
```
#### demangler_gnu
Build using Gradle:
On Linux:
```bash
gradle demangler_gnuLinux64Executable
```
On macOS:
```bash
gradle demangler_gnuOsx64Executable
gradle buildNatives_win64
```
On Windows:
```cmd
gradle demangler_gnuWin64Executable
```
#### sleigh
The sleigh compiler has been ported to Java and integrated with Ghidra.
The native sleigh compiler may still be useful for those who'd like quicker feedback by compiling from the command line.
To build the native sleigh compiler, install bison and flex.
Now, use Gradle:
On Linux:
```bash
gradle sleighLinux64Executable
```
On macOS:
```bash
gradle sleighOsx64Executable
```
On Windows:
```cmd
gradle sleighWin64Executable
```
This will build the decompiler, the demangler for GNU toolchains, the sleigh compiler, and (on Windows only) the PDB parser.
## Run Ghidra from Eclipse
@ -284,15 +259,31 @@ To test it, unzip it where you like, and execute `./ghidraRun`.
Some features of Ghidra require the curation of rather extensive data bases.
These include the Data Type Archives and Function ID Databases, both of which require collecting header files and libraries for the relevant SDKs and platforms.
Much of this work is done by hand.
Until this process is documented, those artifacts can be extracted from an official distribution and combined with your build output.
The archives included in our official builds can be found in the [ghidra-data] repository.
## Building Data Type Archives
TODO
This task is often done manually from the Ghidra GUI, and the archives included in our official build require a fair bit of fine tuning.
From a CodeBrowser window, select File -> Parse C Source.
From here you can create and configure parsing profiles, which lists headers and pre-processor options.
Then, click "Parse to File" to create the Data Type Archive.
The result can be added to an installation or source tree by copying it to `Ghidra/Features/Base/data/typeinfo`.
## Building FID Databases
TODO
This task is often done manually from the Ghidra GUI, and the archives included in our official build require a fair bit of fine tuning.
You will first need to import the relevant libraries from which you'd like to produce a FID database.
This is often a set of libraries from an SDK.
We include a variety of Visual Studio platforms in the official build.
From a CodeBrowser window, select File -> Configure.
Enable the "Function ID" plugins, and close the dialog.
Now, from the CodeBrowser window, select Tools -> Function ID -> Create new empty FidDb.
Choose a destination file.
Now, select Tools -> Function ID -> Populate FidDb from programs.
Fill out the options appropriately and click OK.
If you'd like some details of our fine tuning, take a look at `Ghidra/Features/FunctionID/building_fid.txt`.
# Developing / Building the GhidraDev Plugin
@ -341,3 +332,5 @@ gradle cdtUnpack pyDevUnpack
If you're using BuildShip, simply refresh the Gradle project in Eclipse.
If you're not using BuildShip, re-run `gradle eclipse` and import the new project.
[ghidra-data]: https://github.com/NationalSecurityAgency/ghidra-data

13
Ghidra/Features/Decompiler/src/decompile/cpp/printc.cc
View File

@ -1250,7 +1250,7 @@ bool PrintC::printCharacterConstant(ostream &s,const Address &addr,int4 charsize
/// \brief Push a single character constant to the RPN stack
///
/// For C, a character constant is usually emitted as the character in single quotes.
/// Handle unicode, wide characters, etc.
/// Handle unicode, wide characters, etc. Characters come in with the compiler's raw encoding.
/// \param val is the constant value
/// \param ct is data-type attached to the value
/// \param vn is the Varnode holding the value
@ -1259,10 +1259,17 @@ void PrintC::pushCharConstant(uintb val,const TypeChar *ct,const Varnode *vn,con
{
ostringstream t;
if ((ct->getSize()==1)&&
((val<7)||(val>0x7e)||((val>13)&&(val<0x20)))) // not a good character constant
if ((ct->getSize()==1)&&(val >= 0x80)) {
// For byte characters, the encoding is assumed to be ASCII, UTF-8, or some other
// code-page that extends ASCII. At 0x80 and above, we cannot treat the value as a
// unicode code-point. Its either part of a multi-byte UTF-8 encoding or an unknown
// code-page value. In either case, we print it as an integer.
push_integer(val,1,true,vn,op);
}
else {
// From here we assume, the constant value is a direct unicode code-point.
// The value could be an illegal code-point (surrogates or beyond the max code-point),
// but this will just be emitted as an escape sequence.
if (doEmitWideCharPrefix() && ct->getSize() > 1)
t << 'L'; // Print symbol indicating wide character
t << '\''; // char is surrounded with single quotes

4
Ghidra/Features/Decompiler/src/decompile/cpp/printlanguage.cc
View File

@ -446,6 +446,10 @@ bool PrintLanguage::unicodeNeedsEscape(int4 codepoint)
if (codepoint == 0x3000) {
return true; // ideographic space
}
if (codepoint >= 0xd7fc) { // D7FC - D7FF are currently unassigned.
// D800 - DFFF are high and low surrogates, technically illegal.
return true; // Treat as needing to be escaped
}
return false;
}
if (codepoint < 0xf900) {

11
Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/pcode/PcodeDataTypeManager.java
View File

@ -261,7 +261,16 @@ public class PcodeDataTypeManager {
if (type instanceof Array) {
return buildType(type, size);
}
if (!(type instanceof FunctionDefinition) && type.getLength() <= 0) {
if (type instanceof FunctionDefinition) {
long id = progDataTypes.getID(type);
if (id <= 0) {
// Its possible the FunctionDefinition was built on the fly and is not
// a permanent data-type of the program with an ID. In this case, we can't
// construct a <typeref> tag but must build a full <type> tag.
return buildType(type, size);
}
}
else if (type.getLength() <= 0) {
return buildType(type, size);
}
StringBuilder resBuf = new StringBuilder();

23
Ghidra/Processors/Dalvik/src/main/java/ghidra/dalvik/dex/inject/ConstantPoolDex.java
View File

@ -56,8 +56,9 @@ public class ConstantPoolDex extends ConstantPool {
String classString =
DexUtil.convertTypeIndexToString(dexHeader, fieldIDItem.getClassIndex());
String[] pathArray = DexUtil.convertClassStringToPathArray("", classString);
if (pathArray != null)
if (pathArray != null) {
res.token = pathArray[pathArray.length - 1] + '.' + res.token;
}
}
DataType fieldDT = dexHeader.getDataType(program, fieldIDItem.getTypeIndex());
@ -72,13 +73,16 @@ public class ConstantPoolDex extends ConstantPool {
private String removeUniquifier(String name) {
int len = name.length();
if (len < 10 || name.charAt(len - 9) != '_')
if (len < 10 || name.charAt(len - 9) != '_') {
return name;
}
char matchChar = name.charAt(len - 8);
if (matchChar != '5' && matchChar != 'e')
if (matchChar != '5' && matchChar != 'e') {
return name;
if (name.charAt(len - 7) != '0')
}
if (name.charAt(len - 7) != '0') {
return name;
}
return name.substring(0, len - 9);
}
@ -103,14 +107,19 @@ public class ConstantPoolDex extends ConstantPool {
String classString =
DexUtil.convertTypeIndexToString(dexHeader, methodIDItem.getClassIndex());
String[] pathArray = DexUtil.convertClassStringToPathArray("", classString);
if (pathArray != null)
if (pathArray != null) {
namespaceString = pathArray[pathArray.length - 1];
}
}
if (namespaceString != null)
if (namespaceString != null) {
res.token = namespaceString + '.' + res.token;
}
}
res.tag = ConstantPool.POINTER_METHOD;
FunctionDefinitionDataType funcDef = new FunctionDefinitionDataType(res.token, dtManager);
// The FunctionDefinition is constructed on the fly, essentially as an anonymous type
// We use an internal naming scheme involding the the methodID to avoid name collisions
String defName = res.token + '_' + Integer.toHexString(methodID);
FunctionDefinitionDataType funcDef = new FunctionDefinitionDataType(defName, dtManager);
res.type = new PointerDataType(funcDef);
res.hasThisPtr = !isStatic;

11
README.md
View File

@ -4,13 +4,10 @@ Ghidra is a software reverse engineering (SRE) framework created and maintained
In support of NSA's Cybersecurity mission, Ghidra was built to solve scaling and teaming problems on complex SRE efforts, and to provide a customizable and extensible SRE research platform. NSA has applied Ghidra SRE capabilities to a variety of problems that involve analyzing malicious code and generating deep insights for SRE analysts who seek a better understanding of potential vulnerabilities in networks and systems.
This repository contains the framework and core features.
More often than not, you can develop extensions and scripts without modifying the core framework.
Nevertheless, we appreciate all your contributions.
Try out the GhidraDev plugin for Eclipse, which is part of the distribution package, to get started developing.
The release can be downloaded from our [project homepage][project].
If you would like to contribute to the framework and core feature set, please take a look at our [contributor guide][contrib] to see how
you can participate in this open source project.
To start developing extensions and scripts, try out the GhidraDev plugin for Eclipse, which is part of the distribution package. The full release build can be downloaded from our [project homepage][project].
This repository contains the source for the core framework, features, and extensions.
If you would like to contribute, please take a look at our [contributor guide][contrib] to see how you can participate in this open source project.
If you are interested in projects like this and would like to develop this, and
other cybersecurity tools, for NSA to help protect our nation and its allies,

2
gradleScripts/jacoco.excludes.src.txt
View File

@ -50,8 +50,6 @@ ghidra/GhidraThreadGroup*
ghidra/HelpAdapter*
ghidra/ClassSearcherStatusReportingTaskMonitor*
ghidra/app/plugin/debug/**
generic/platform/OSXAdapter*
// Old/deprecated APIs
ghidra/app/program/database/oldfunction/**