From 1090e35df3569f08ae3751a4508fb6d9a8c2d2c9 Mon Sep 17 00:00:00 2001 From: emteere <47253321+emteere@users.noreply.github.com> Date: Mon, 3 Jun 2024 13:23:48 -0400 Subject: [PATCH] GP-0 WhatsNew for 11.1 release --- .../src/global/docs/WhatsNew.html | 250 ++++++++++-------- 1 file changed, 137 insertions(+), 113 deletions(-) diff --git a/Ghidra/Configurations/Public_Release/src/global/docs/WhatsNew.html b/Ghidra/Configurations/Public_Release/src/global/docs/WhatsNew.html index ed8ac86ac1..5d01f3292c 100644 --- a/Ghidra/Configurations/Public_Release/src/global/docs/WhatsNew.html +++ b/Ghidra/Configurations/Public_Release/src/global/docs/WhatsNew.html @@ -45,148 +45,172 @@
This release includes new features, enhancements, performance improvements, quite a few bug fixes, and many pull-request contributions. Thanks to all those who have contributed their time, thoughts, and code. The Ghidra user community thanks you too!
-Ghidra 11.0 is fully backward compatible with project data from previous releases. - However, programs and data type archives which are created or modified in 11.0 will not be useable by an earlier Ghidra version.
+Ghidra 11.x is fully backward compatible with project data from previous releases. + However, programs and data type archives which are created or modified in 11.x will not be usable by an earlier Ghidra version.
-This distribution requires JDK 17 to run, and provides Linux x86-64, Windows x86-64, and macOS x86-64 native components. - If you have another platform or wish to use a newer JDK, please see the - Ghidra Installation Guide for additional information.
+This distribution requires at minimum JDK 17 to run, but can also run under JDK 21.
-NOTE: Ghidra Server: The Ghidra 11.0 server is compatible with Ghidra 9.2 and later Ghidra clients. Ghidra 11.0 +
NOTE: Each build distribution will include native components (e.g., decompiler) for at least one platform (e.g., Windows x86-64). + If you have another platform that is not included in the build distribution, you can build + native components for your platform directly from the distribution. + See the Ghidra Installation Guide for additional information.
+ +IMPORTANT: To use the Debugger, you will need Python3 (3.7 to 3.12 supported) installed on your system.
+ +NOTE: Ghidra Server: The Ghidra 11.x server is compatible with Ghidra 9.2 and later Ghidra clients. Ghidra 11.x clients are compatible with all 10.x and 9.x servers. Although, due to potential Java version differences, it is recommended that Ghidra Server installations older than 10.2 be upgraded. Those using 10.2 and newer should not need a server upgrade.
NOTE: Any programs imported with a Ghidra beta version or code built directly from source code outside of a release tag may not be compatible, and may have flaws that won't be corrected by using this new release. Any programs analyzed from a beta or other local master source build should be considered - experimental and re-imported and analyzed with a release version. Programs imported with previous release versions should upgrade correctly through various - automatic upgrade mechanisms. Any program you will continue to reverse engineer should be imported fresh with a release version or a build you trust with the - latest code fixes.
- - - -A major new feature called BSim has been added. BSim can find structurally similar functions in (potentially large) collections of binaries or object files. - BSim is based on Ghidra's decompiler and can find matches across compilers used, architectures, and/or small changes to source code.
-As you've reverse engineered software, you've likely asked the following questions:
+Programs imported with previous release versions should upgrade correctly through various + automatic upgrade mechanisms. However, there may be improvements or bug fixes in the import and analysis process that will provide better results than prior + Ghidra versions. You might consider comparing a fresh import of any program you will continue to reverse engineer to see if the latest Ghidra + provides better results.
+ATTENTION: Please delete and re-import the default Debugger tool!
+ +We are introducing a new debugger connection system called Trace RMI. This is replacing the older system, + which we are calling the Recorder system.
+ +
The most noticeable difference will be a new menu for launching targets. It is very similar to the previous system, but with some key differences:
BSim is intended to help with these questions (and others) by providing a way to search collections of binaries for similar, but not necessarily identical, functions.
+The next most noticeable difference will be the replacement of the Interpreter window with the Terminal window. This is a proper VT-100 + terminal emulator, so the experience will be much like, if not identical to, how you'd debug in a plain terminal, except embedded into and integrated with Ghidra. + Some notable improvements that brings:
+BSim can compare functions within a binary, within a collection of binaries or object files in a project on a local system, or within a large collection of binaries - utilizing a PostgreSQL or an Elasticsearch server. Using BSim locally does not require setting up a PostgreSQL or Elastic server or having administrator access.
- -There is a BSim tutorial that walks through use of BSim locally. Using BSim locally and the tutorial is the best way to try out BSim before deciding if you need to set up a server.
- -GhidraGo is an experimental feature that adds integration support for Ghidra URL's and Ghidra Tools. The main use of GhidraGo is embedding hyperlinks within web pages - to pre-ingested programs within a Ghidra multi-user repository. Clicking on the hyperlink causes Ghidra to display the previously ingested program. - No data other than the Ghidra URL is transferred to Ghidra, and no socket is open within Ghidra listening for commands. GhidraGo must be enabled by - installing a plugin in the Ghidra project manager, and must also be configured as a protocol handler in your web browser. GhidraGo is not setup or enabled by default. - For details on setting up GhidraGo, please see the included GhidraGoREADME.html or search for GhidraGo within help. -
+You may also notice the replacement of the Debugger Targets window with the Connection Manager window, and the replacement + of the Objects window with the Model window. These are operationally very similar to their previous counterparts.
+For Power Users: The launchers are just shell scripts on Linux and macOS, and batch files on Windows. We have provided plugins + for integrating with GDB, LLDB, and the Windows Debugger. So long as your target works with one of these debuggers, orchestrating + another kind of target is mostly a matter of creating a new shell script. This is usually accomplished by using the most similar + one as a template and then trying it out in Ghidra. When errors occur, Ghidra will inform you of what progress it made before it + failed, and the Terminal should display any error messages produced by your script.
+ +For Developers: Developers may notice that debugger integration is now all done using Python 3. + We have specified a new protocol we call Trace RMI, which provides client access to Ghidra's trace databases over TCP. + It uses protobuf and is substantially simpler than the previous GADP protocol. We have provided the client implementation in + Python 3. Existing integrations can be fairly easily extended, if necessary. For example, see the support for Wine we included in our GDB plugin.
+ +If you wish to integrate a completely new debugger, and it has a Python 3 API, then things are relatively straightforward, so long as + the debugger provides the events and information that Ghidra expects. Use an existing plugin as a template or reference and have fun. + If the new debugger does not have Python 3 bindings, the protobuf specification is available, so the client can be ported, if necessary.
+ +IMPORTANT: To use the new Trace RMI system, you will need Python3 (3.7 to 3.12 supported) installed on your system. + Additional setup may be required for each type of debug connection. Press F1 in the debug connector's launch dialog + for more information.
+ +Overall, we believe this a substantially more approachable system than our previous DebuggerObjectModel SPI used in the Recorder system.
+ + +GhidraGo is an experimental feature that adds integration support for Ghidra URL's and Ghidra Tools. GhidraGO can now process GhidraURL's that + locate folders within a project instead of only programs. For example a remote GhidraURL locating a project folder will open a read only view of + the repository in the front end tool and select the folder from the URL. If the GhidraURL refers to a folder in the currently open + active project, then the folder is selected within the active project's view instead of a read only view. +
+ +The PDB data type processing changes from release 11.0 have been further enhanced, simplifying the processing model and reducing the number of datatype + conflicts. The algorithm for choosing the primary symbol at an address has been improved to provide the richest possible information. The PDB Universal + Analyzer has been split into multiple analyzers so that PDB function processing can follow interim analyzers that specialize in finding code. + Lastly, the Load PDB Task has been improved to schedule appropriate follow-on analyzers that are selected in the Analysis Options.
+Auto Version Tracking has been sped up, made customizable, and improved to find more matches. The mechanism to identify good matches from duplicate match - sets has been improved and sped up. Implied matches are now created and will be applied if the vote minimum and conflict maximum limits are met. In addition, - the ability to choose which correlators are run as well as setting the options of most correlators has been added. The Auto Version Tracking script has been - updated to prompt for all options in a single dialog. The script now works in headless mode and an example prescript to allow - setting of options in headless mode is included.
- -Version Tracking can also use the new BSim function matching capability in a new correlator called the BSim Correlator. Auto Version Tracking does not use the new BSim Correlator yet.
+Version Tracking Session files may now be added to a shared project repository. Once a version tracking file has been checked in to a project, + it must be checked out for exclusive access. For more information, see help found in the Version Tracking's + Session Wizard help for more information.
-
The function comparison window, used by Version Tracking and BSim, has been overhauled a bit: - -
A new multi-option script dialog has been added to the scripting API that can present and get all user inputs a script needs in one dialog. - The askValues() scripting method replaces the cumbersome process of prompting the user for each input separately.
- -NOTE: Prior to adding a pre-existing VT Session to a shared project repository, it is highly recommended that it first be re-opened + and saved. This will upgrade the VT Session internal version to prevent its use with older versions of Ghidra which will not respect + the exclusive checkout requirement.
-Initial support for Rust compiled binaries, mainly demangling of Rust method names and Rust in DWARF information, has been added. In addition, Rust strings are marked up so that the - decompiler will display Rust strings correctly. There is more work to do, especially with mapping Rust parameter passing. Custom storage may be required in some instances.
+Mach-O support continues to improve, adding support for new features as well as filling in some gaps that existed for several years. + The latest dyld_shared_cache files use a new format for pointer fixups, which Ghidra now supports. A new GFileSystem has also been + implemented to import and/or extract individual Mach-O binaries from Mach-O "file sets" (i.e., kernelcache). A second new GFileSystem + has been added which can extract Apple LZFSE-compressed files. Other improvements have also been made to provide more complete markup of Mach-O load commands.
-Golang reverse engineering within Ghidra is much improved by: -
Initial support for binaries written in the Swift Programming Language has been added. The new support relies on the native Swift demangler being + present on the user's system. Swift is automatically bundled with XCode on macOS, and can be optionally installed on Windows and Linux. + See the "Demangler Swift" analyzer options for more information. Type information gathered from the demangled Swift symbol names is used to + create corresponding Ghidra data types. This currently works for Swift primitives and structures, but more work needs to be done to include + classes and other advanced data types. Swift-specific calling conventions are also applied to demangled Swift functions.
+ +There have been many improvements to keyboard only actions and navigation in Ghidra. These changes will be welcome for those who + prefer to use the keyboard as much as possible and those needing better accessibility. Improvements include:
+A new action in the Search menu, Search -> For Encoded Strings..., can help find and create strings encoded in alternate character sets and alphabets. - Valid strings are based on runs of bytes that would be valid in a particular character set and alphabet. There are currently no additional models for defining valid words within other languages.
-The CaRT file format is now supported. The CaRT format is used to store and transfer malware along with metadata about - the malware in a neutered form such that it cannot be executed. It is encrypted so anti-virus software will not flag the file under analysis.
- -Headless importing of binaries from container files, such as .zip files, with multiple embedded files is now possible. This includes loading referenced .dll and .so files also - found within the container file.
- -The Headless Analyzer can now recurse into supported GFileSystem container files when a recursion depth of one or more is specified on the command line.
- -Support for the Mach-O binary file format has continued to receive updates. Improvements have been made to library linking and loading, as well as thunk creation. Additionally, dyld_shared_cache - components extracted from Ghidra's DyldCacheFileSystem can now be added together on-demand with the Add To Program feature. Broken references can be automatically resolved by right-clicking on them - and clicking References -> Add To Program.
- -The PDB data types processing has been changed to use a resolve-as-you-go model. The change eliminates the dependency graph and reduces the memory footprint required to load all - PDB types. The change allows larger PDB's to load successfully and improves the accuracy of some data types.
- -Overlay spaces now support multiple memory blocks in the same overlay. After creating the initial memory - block as an overlay, the new overlay memory space will become available when adding new memory blocks. All overlay memory blocks can - be manipulated in the same way as normal memory blocks. The new feature is useful when analyzing binaries - meant to run on multiple processors with tasks running on each processor in their own overlapped virtual memory space such as an RTOS.
- -Support for the Loongson processor architecture has been added. All known instructions should disassemble. However semantics for a large number of instructions use pseudoOp calls currently.
- +Support for the squashfs filesystem has been added.
+A new wildcard assembler API has been added that can generate all possible variants of an instruction with a variety of wildcards for operands. + Two new scripts, FindInstructionWithWildcard and WildSleighAssemblerInfo, demonstrate how to use the API. + For more information, see help and search for Wildcard Assembler. + +
A new Runtime Information dialog has replaced the Show VM Memory dialog. The dialog contains more information + which can aid in debugging, including version information, classpath, defined properties, environment variables, and more.
+ +The GhidraDev Eclipse plugin has a new wizard for importing an existing Ghidra module source directory. This will work best with Ghidra module projects + created against Ghidra 11.1 or later.
+ +Finding references to fields within a structure has been greatly improved. Previously many references to the field would be missed if they occurred within + functions calling external functions using the structure, or when the field was used only in local variables dynamically generated by + the decompiler.
+ +Golang versions 17 thru 22 are now supported.
+ +DWARF5 debug format is now supported. In addition, DWARF line number information processing has been incorporated into the base DWARF analyzer and the + separate DWARF line number analyzer has been removed.
+Numerous other new features, improvements, and bug fixes are fully listed in the ChangeHistory file.