Writing a Total Commander plug-in in Visual Basic (or C#)

• Download binary - 1.07 MB
• Download source - 2.3 MB

Introduction

Windows Explorer is not enough. People who have the same feeling as me can choose of several alternative file managers. One of the most popular is Total Commander - a shareware file manager for Microsoft Windows. It has the classic two-panes view known from Norton Commander from old DOS times, and supports many file operations. One of the important features of Total Commander is the ability to be extended via plug-ins. This article describes how to write a Total Commander plug-in in a managed language (Visual Basic or C#), in general, and with special focus on File System plug-ins (WFX).

Plug-in architecture

Total Commander itself is written in the Delphi (Object Pascal) language (unmanaged). The Delphi compiler produces C-style executables and libraries. The plug-in interface is designed for plug-ins written in C/C++. The author provides C header files for plug-in authors. There are four plug-in types:

Content plug-in (WDX)

Allows Total Commander to show additional details of files (like ID3 track name, or Exif camera model) and use them for search or renaming.

File System plug-in (WFX)

Allows Total Commander to access other file systems like Linux-formatted partitions, device file systems, or FTP servers.

Lister plug-ins (WLX)

Allows Total Commander to show contents of files of various types (like MP3, HTML etc.).

Packer plug-ins (WCX)

Allows Total Commander to access - show, extract, pack, and manipulate contents of various types of archives (like CAB or MSI).

Each plug-in is created as a C DLL library which exports a defined set of functions. The library has the extensions wdx/wfx/wlx/wcx instead of DLL depending on which type of plug-in it represents. The set of functions that a plug-in must export is defined by the Total Commander plug-in interface. There are only a few compulsory functions for each type of plug-in, and then there are plenty of optional functions. The compulsory functions provide the very basic necessary functionality of a plug-in. For a file system plug-in, only four functions providing initialization and the list of files and directories are compulsory. Then, there are optional functions for downloading, uploading, deleting, renaming etc. Of course, there are no limitations on the Total-Commander-unrecognized functions a plug-in can export. As newer and newer versions of Total Commander are developed, the set of supported plug-in functions grows. So, when using a plug-in designed for a newer version of TC with an older one, some functions are never called.

Writing a Total Commander plug-in in C++ is an easy task. It's an easy task in any language that can export functions in a C-like way, like Delphi or PowerBasic. But, managed .NET languages (like Visual Basic or C#) cannot export functions this way. .NET can export COM objects, but not those Win32-API-like functions, even though .NET can import them using DllImportAttribute and Visual Basic has a special syntax for importing DLL functions. As far as I know, it is even technically impossible to export those functions from a managed assembly because .NET does not provide a way to place functions on a static address - .NET functions get their addresses when they are JIT-compiled. Another limitation, especially for Visual Basic, is the lack of support for pointers widely used in TC plug-in functions. OK, the simple answer to the question, "Can I write a Total Commander plug-in in Visual Basic (C#)?" is "No". As you probably guessed, "No" is not the answer I put up with.

There is one special language in .NET that can combine managed and unmanaged code in one assembly, that can export Win32-style functions, that can define global functions, and that can work with pointers - it's C++/CLI. So, my solution to write a managed Total Commander plug-in is:

1. Write an interface between Total Commander and the managed code in C++/CLI.
2. Write the plug-in in any managed language.

The C++/CLI interface is called by the Total Commander, and it converts all the ugly data types from C++ like char* to nice managed types like String. The managed plug-in, then, does its work and returns what it should return. The C++/CLI interface converts the managed return value to an unmanaged one, and passes it back to Total Commander in the requested fashion.

If an interface is implemented simply as written above, you must write (copy and paste) the interface again and again for each plug-in you'll write. What I wanted was some general purpose solution. So, I've created a C++/CLI assembly that contains some support classes and structures for passing data between managed and unmanaged code, and above all, it contains the plug-in base class the actual plug-in implementations are derived from. So, the way the plug-in is implemented is fully object oriented as it is common in .NET. The basic parts of the Total Commander plug-in managed framework are:

C++/CLI unmanaged ↔ managed interface (the Tools.TotalCommander assembly)

The plug-in base class and the support classes and structures are contained in this assembly.

Plug-in implementation

The managed assembly (DLL) that implements the plug-in. It can even implement more than one plug-in or plug-ins of different types. It can be written in any managed language that can derive from the plug-in base class.

Plug-in assembly

A small assembly written in C++/CLI. This assembly represents the plug-in from the Total Commander point of view. It has the required extension (wdx/wfx/wlx/wcx), and it exports all the necessary functions. It initializes the plug-in instance, and then it simply calls the plug-in instance function whenever Total Commander calls the plug-in function. This assembly is generated by the Total Commander Plug-in Builder from the plug-in implementation.

Total Commander Plug-in Builder

The command line tool that builds the plug-in assembly from the predefined template using information from the plug-in implementation. Especially, it ensures that the right type of plug-in is built and that only functions implemented by the plug-in-implementing class are exported by the plug-in library.

Interface between Total Commander and managed code

As written above, this assembly is written in C++/CLI, and performs marshaling between unmanaged (Total Commander) and managed (plug-in implementation) code. It contains some support classes and structures, some of them are visible from managed code. It also defines several attributes used to specify how the Plug-in Builder builds the plug-in assembly. In fact, this assembly contains only a very little portion of the unmanaged code - only the definition of the unmanaged structures imported from Christian-Ghisler-provided header files (and those header files include a few Windows SDK header files). But, the code in this assembly deals with those unmanaged types and with pointers and C++-like strings (char*). It's something we avoid in C# and can't do in Visual Basic.

The plug-in (abstract) base class simply contains non-virtual (not overridable in VB) functions accepting and returning unmanaged types, and then it contains virtual (overridable in VB) functions overridden by the actual plug-in implementation. Virtual functions accept and return managed and CLS-compliant types. Non-virtual functions convert the parameters from unmanaged to managed types, and pass them to the virtual function. When the virtual function returns, its return value (and out parameter values) is converted from managed to unmanaged and passed to the caller. The caller is actually a global function in the plug-in assembly which passes the values back to the Total Commander. A virtual function differs in behavior from a non-virtual one. For example, exceptions are used instead of error return codes, and return values instead of output parameters (sometimes, output parameters cannot be avoided - multiple return values).

In a file system plug-in, the mapping between non-virtual and virtual functions is usually 1:1. For almost each non-virtual function (starting with the FS prefix), a corresponding virtual function exists. Virtual functions for compulsory functions have no implementation, which effectively makes the plug-in author to implement them in the derived class. Optional functions have default implementations throwing NotSupportedException. It is ensured that the Total Commander never calls an optional function which is not overridden in the plug-in class because such a function is not exported by the plug-in assembly (the Total Commander Plug-in Builder does not generate exports for them). From an object oriented point of view, it can be determined that the actual implementation of an optional function does nothing but throws NotSupportedException by the MethodNotSupportedAttribute applied on the method.

Sometimes, the interface provides a little bit higher level of abstraction than the unmanaged Total Commander plug-in interface. It does not use handles, but actual objects - bitmaps and icons.

The plugin assembly

The plug-in assembly contains code responsible for creating an instance of a plug-in, and it actually exports the functions to unmanaged environments and passes function calls from the Total Commander to the plug-in abstract base class. As the plug-in assembly is generated by the Total Commander Plug-in Builder from a template using information from the plug-in implementation, it is customized for the actual plug-in it represents - and it always represents only one plug-in. In case the plug-in implementation assembly contains more plug-ins, more plug-in assembles are generated.

The most tricky part of the work in the plug-in assembly is the assembly binding. The plug-in assembly has references to Tools.TotalCommander and to the plug-in implementing assembly. Tools.TotalCommander refers to Tools, and the plug-in implementing the assembly may refer to any assembly - locally copied or GAC. Total Commander plug-ins usually reside in subfolders of the subfolder plugins of the Total Commander installation folder. And now, problems arise: the plug-in assembly is loaded to the totalcmd.exe process. Totalcmd.exe resides two or more folders above the plug-in assembly. The plug-in assembly refers to the plug-in implementation assembly and Tools.TotalCommander, neither of them is in GAC. By default, .NET looks for references in the same directory as the process was started in. Subfolders are not examined. So, references are not found and the plug-in crashes. Total Commander can recover from it, but the plug-in is not loaded and it does not work. The only assembly that is correctly loaded is the plug-in assembly, because it is loaded as part of something that seems to be an unmanaged Win32-API-style DLL. So, the plug-in assembly must ensure that references will be searched where they lie. We can solve the issue in several ways:

Place all the assemblies in the same directory as totalcmd.exe

It is not a good idea as many, possibly conflicting, files will be in the Total Commander installation directory. This may lead to chaos.

Place all the necessary assemblies in GAC

This is also not a very good solution. First, it puts additional demands to the plug-in installation process and requires administrator privileges. It prevents the plug-in from being carried with Total Commander when it is installed on a flash drive. And, single-purpose assemblies as the Total Commander plug-in implementation assembly should not be in the GAC.

Intercept assembly resolution

We can handle the event AppDomain.AssemblyResolve which is raised when assembly resolution fails. The handler of this event can load the assembly and return it. Problems will arise when multiple managed plug-ins are used with Total Commander. Plug-ins can use different versions of Tools.TotalCommander, or plug-ins need not be based on this framework. This assembly resolution can effectively destroy other managed plug-ins.

Load each plug-in into a separate application domain

This is the way I've finally chosen. Only the plug-in assembly is loaded to the default domain. It creates another domain and sets its base directory to the directory it is located in. Then, it creates an instance of the helper class in the newly created application domain. The class creates an instance of the plug-in class - it is resolved correctly, because all references are in its base directory. Assemblies loaded by the plug-in does not interfere with other plug-ins because application domains are separated. There is only some overhead because TC calls the global function in the plug-in assembly, it calls the function in the assembly domain helper, it calls the function in the plug-in helper, it calls non-virtual functions in the plug-in class, and it finally calls the virtual function in the plug-in class.

Total Commander Plug-in Builder

A command line tool that creates a plug-in assembly for each plug-in class in the plug-in implementation assembly. It can be invoked from the command line, or it can be used programmatically. It needs access to the C++/CLI compiler, vcbuild.exe. It is written in Visual Basic. The best way of using the Total Commander Plugin Builder is to have it in the post-build event in Visual Studio.

It enumerates all types in the plug-in implementation assembly, and for those that represent Total Commander, the plug-in generates a plug-in assembly. While generating a plug-in assembly, the plug-in class is examined to determine which plug-inplugin functions are implemented (overridden) by the plug-in class. Methods that are not implemented are not generated in the plug-in assembly. It is achieved simply by writing several C++ preprocessor #defines to control how the plug-in assembly will be compiled. Same way, the name of the class to create an instance of it is specified. A reference to the plug-in implementation assembly is set by the #using C++ directive. the Total Commander Plug-in Builder also examines certain attributes of the plug-in implementation assembly and the plug-in class to set the plug-in assembly attributes and to refine the generation behavior.

The code

OK, I'm not gonna re-type all the code here. Download the attached example. Only a few interesting parts of the code:

Creation of the application domain

The following code snippet in C++/CLI shows how the application domain is created:

namespace Tools{namespace TotalCommanderT{
    extern bool RequireInitialize;
    extern gcroot<AppDomainHolder^> holder;
    //PluginInstanceHolder class keeps plugin instance
    PluginInstanceHolder::PluginInstanceHolder(){
        this->instance = TC_WFX;
        //TC_WFX ide C++ preprocessor macro defined 
        //to somethig like gcnew MyPluginClass()
    }
    //AppDomainHolder keeps AppDomain instance
    AppDomainHolder::AppDomainHolder(){
        this->holder = gcnew PluginInstanceHolder();
    }
    //Global function initialize is called by plugin functions FsInit 
    //and FsGetDefRootName which can be called 
    //as first call to plugin by Total Commander
    void Initialize(){
        if(!RequireInitialize) return;
        RequireInitialize = false;
        PluginSelfAssemblyResolver::Setup();
        AppDomainSetup^ setup = gcnew AppDomainSetup();
        Assembly^ currentAssembly = Assembly::GetExecutingAssembly();
        setup->ApplicationBase = IO::Path::GetDirectoryName(currentAssembly->Location);
        AppDomain^ pluginDomain = AppDomain::CreateDomain(PLUGIN_NAME,nullptr,setup);
        AppDomainHolder^ iholder = 
          (AppDomainHolder^)pluginDomain->CreateInstanceFromAndUnwrap(
              currentAssembly->CodeBase,AppDomainHolder::typeid->FullName);
        Tools::TotalCommanderT::holder = iholder;
    }
}}

PluginSelfAssemblyResolver is a simple helper class that allows the resolution of the assembly itself when it cannot be found by .NET. It contains only two functions:

namespace Tools{namespace TotalCommanderT{
    Assembly^ PluginSelfAssemblyResolver::OnResolveAssembly(Object^ sender, 
                                          ResolveEventArgs^ args){
        AssemblyName^ name = gcnew AssemblyName(args->Name);
        if(AssemblyName::ReferenceMatchesDefinition(name, 
                thisAssembly->GetName())) return thisAssembly;
        else return nullptr;
    }
    inline void PluginSelfAssemblyResolver::Setup(){
        AppDomain::CurrentDomain->AssemblyResolve += 
          gcnew ResolveEventHandler( PluginSelfAssemblyResolver::OnResolveAssembly );
    }
}}

Definition of plug-in functions

Optional as well as compulsory Total Commander plug-in functions are wrapped in #ifdef-#endif blocks. The corresponding #defines for those blocks are written by the Total Commander Plug-in Builder to the define.h file. Due to the architecture using the application domains, those functions are in the plug-in assembly three times with identical signature and similar body; I've extracted them to a separate file, wfxFunctionCalls.h. This file is included at three different places with several C++ preprocessor #defines to control how it is compiled. Each function is defined like this:

#ifdef TC_FS_INIT
    TCPLUGF int FUNC_MODIF FsInit(int PluginNr,tProgressProc pProgressProc, 
                tLogProc pLogProc,tRequestProc pRequestProc){
        return FUNCTION_TARGET->FsInit(PluginNr,pProgressProc,pLogProc,pRequestProc);
    }
#endif

TCPLUGF is defined as empty (not used). Once I thought about using __declspec(dllexport) to export functions. Lately, I've switched to a separate Exports.def file. FUNC_MODIF is either __stdcall or the class name (AppDomainHolder:: or PluginInstanceHolder::). __stdcall is used for exported functions; internal calls use the managed calling convention. Finally, FUNCTION_TARGET is the instance to call the function on. It is Tools::TotalCommanderT::holder in exported (global) functions, this->holder in AppDomainHolder, and this->instance in PluginInstanceHolder.

WfxFunctionCalls.h is included like this:

#define TCPLUGF
#define FUNC_MODIF AppDomainHolder::
#define FUNCTION_TARGET this->holder
#include "FunctionCalls.h"

I know, it may be more comprehensible typing it thrice. But, so many functions are typed thrice - I'm really lazy, and besides, when some change is needed, it needs to be done only once (in the plug-in assembly template, then in the plug-in base class and in the common header file, and ...).

Marshalling

Marshalling from unmanaged to managed code is quite simple. The Total Commander plug-in interface uses several structures and constants. For structures, I've created managed counterparts, and before passing an object to managed code, the structure is converted to a managed one. Before a structure is returned to unmanaged code, it is converted back. Constant values are represented by managed enumeration values, and are simply cast. Something very often passed between the Total Commander and a plug-in are strings. Total Commander passes and accepts strings as char* (sometimes, char[MAX_PATH]) - always null-terminated. Marshaling those values from unmanaged to managed code is easy, because System.String has a constructor that accepts char* (System.SByte*).

Note: In the current version, Total Commander neither passes to plug-ins nor accepts from them Unicode strings (wchar_t*) although a few Win32 API structures used by Total Commander are declared as Unicode. Total Commander uses the current system encoding. Unicode support will be available in one of future versions of Total Commander. This is not a limitation of my framework but of Total Commander itself.

Passing a string to unmanaged code is a little more tricky. It is possible to enumerate all the characters of a string easily in .NET. But, those characters are Unicode code points. They must be converted to default system encoding values. Finally, I've created my own StringCopy functions:

namespace Tools{namespace TotalCommanderT{
    void StringCopy(String^ source, char* target, int maxlen){
        if(source == nullptr)
            target[0]=0;
        else{
            System::Text::Encoding^ enc = System::Text::Encoding::Default;
            cli::array<unsigned char>^ bytes = enc->GetBytes(source);
            for(int i = 0; i < bytes->Length && i < maxlen-1; i++)
                target[i]= bytes[i];
            target[source->Length > maxlen-1 ? maxlen-1 : source->Length] = 0;
        }
    }
    void StringCopy(String^ source, wchar_t* target, int maxlen){
        StringCopy(source,(char*)(void*)target,maxlen);
    }
}}

The second function simply treats a wchar_t* as char*, see note above.

The function encodes a string using the default system encoding and then copies the encoded bytes to an unmanaged buffer (maximally, maxlen - 1 characters). The character after the last used character is set to nullchar.

Note: I'm not sure if the default encoding will behave correctly in systems where the default encoding is multibyte (e.g., Chinese). I hope for Unicode implementations in future version of TC.

Sample plug-in

The sample plug-in is the simplest that can be written. It simply accesses the local file system. Christian Ghisler provides such an example plug-in in C++. Mine is written in Visual Basic. It shows how to utilize the Managed Total Commander Plug-in Framework.

Notes

• Both, Tools.TotalCommander and the sample plug-in uses (not very extensively) my open source library Tools. It can be downloaded from codeplex.com/Tools and also the latest version of the plug-in framework.
• Edit the Post-Build event and the debug command line of the wfx sample project to reflect the actual location of the Total Commander before building. You need to run Visual Studio with elevated privileges on Vista, or have write rights to the target directory (C:\Program Files\totalcmd\plugins\Test\wfx sample\Debug\).
• The example source code is only example source code. The project has commented out several pre-build and post-build events needed when larger modifications are done! Uncommenting the events requires several utilities to perform the events. Download the full source from Codeplex.

Known issues

• The current version of the framework supports only file system plug-ins (WFX). Other types of plug-ins will be supported in future.
• The current version of framework does not support Unicode. This is due to the Total Commander limitation for plug-ins. It will be hopefully removed in one of the near-future versions of Total Commander.
• The only way the sample plug-in can access UNC paths is when the user manually changes to a UNC directory from the Total Commander command line.
• The properties window of a file/folder in the sample plugin is not modal to the Total Commander main window. This is not a limitation of the plug-in framework. I use Win32 API to show the properties dialog, and I have no idea how to make it modal.

License

The plug-in framework, the plug-in builder, the sample plug-in, as well as any other code in this article is released under the Open Source license at codeplex.com/Tools.

History

• 2009-03-08 - Initial release.
• 2009-03-08 - Updated source (missing statement in AssemblyResolver.cpp).