Delphi - Retrieve email using Gmail/G Suite OAuth 2.0 + IMAP4 protocol in background service (service account)

By default, you need to enable ” Allowing less secure apps” in Gmail/G Suite, then you can retrieve email with user/password IMAP4 authentication.

However Google will disable traditional user authentication in the future, switching to Google OAuth is strongly recommended now.

Installation

Before you can use the following sample codes, you should download the EAGetMail Installer and install it on your machine at first. Full sample projects are included in this installer.

Add reference

To better demonstrate how to retrieve email and parse email, let’s create a Delphi Standard EXE project at first, then add a TButton on the Form, double-click this button. It is like this:

Delphi console project

To use EAGetMail ActiveX Object in your Delphi project, the first step is “Add Unit file of EAGetMail to your project”. Please go to C:\Program Files\EAGetMail\Include\delphi or C:\Program Files (x86)\EAGetMail\Include\delphi folder, find EAGetMailObjLib_TLB.pas, and then copy this file to your project folder.

// include EAGetMailObjLib_TLB unit to your Delphi Project
unit Unit1;

interface

uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, EAGetMailObjLib_TLB, StdCtrls;

Then you can start to use it in your Delphi Project.

You can also create EAGetMailObjLib_TLB.pas manually by Delphi like this:

  • Delphi 7 or eariler version

    First of all, create a standard delphi project: select menu Project -> Import Type Library, checked EAGetMail ActiveX Object and click Create Unit. Then include EAGetMailObjLib_TLB in your project.

    add reference in Delphi
  • Delphi XE or later version

    First of all, create a standard delphi project: select menu Component -> Import component... -> Import a type library -> checked EAGetMail ActiveX Object, have Generate Component Wrapper checked and click “Create Unit”. Then include EAGetMailObjLib_TLB in your project.

Google Service Account

Normal OAuth requires user input user/password in Web Browser. Obviously, it is not suitable for background service. In this case, you should use google service account to access G Suite email service without user interaction. Service account only works for G Suite user, it doesn’t work for personal Gmail account.

Create project in Google Developers Console

To use “G Suite Service Account OAuth” in your application, you should create a project in Google Developers Console at first.

Important

You can use any google user to create service account, it doesn’t require service account owner is a user in G Suite. But G Suite administrator must authorize service account in G Suite Admin Console to access user mailbox.

Create service account in current project

  • Click "Credentials" -> "Manage service accounts"

    manage service account in google developers console
  • Click "CREATE SERVICE ACCOUNT"

    create service account in google developers console
  • Input a name for your service account, click "CREATE"

    create service account in google developers console step 1
  • In "Service account permissions", select "Project" -> "Owner" as role

    create service account in google developers console step 2
  • In "Grant users access to this service account", keep everything default and click "DONE"

    create service account in google developers console step 3

After service account is created, you should enable "Domain-wide delegation" and create service key pair to access G Suite user mailbox.

Enable “Domain-wide delegation” and create service key

  • Go back to your service account, click "Edit" -> "SHOW DOMAIN-WIDE DELEGATION", check "Enable G Suite Domain-wide Delegation", input a name for product oauth consent, click "Save".

    Enable G Suite Domain-wide Delegation
  • Go back to your service account again, click "Create Key", you can select "p12" or "json" key type, both can work well, then you will get a file which contains private key, save the file to local disk.

    Now you have created service account with key pair successfully. You can use created private key in your codes to request "access token" impersonating a user in G Suite.

    create service key
  • To access user data in G Suite, you must get authorization from G Suite administrator. You should go to service accounts list, click "View Client ID" like this:

    google oauth client id
  • Then copy your “Client ID” and service account email address, forward it to G Suite administrator for authorization.

    google oauth serivce client id

Enable Gmail API

Enable Gmail API in "Library" -> Search "Gmail", then click "Gmail API" and enable it. If you use Gmail API protocol to send email, you should enable this API, if you use SMTP protocol, you don’t have to enable it.

enable Gmail API

Authorize service account by G Suite administrator

To use service account to access user mailbox in G Suite, G Suite Administrator should authorize specified service account at first.

Important

Important Notice: You can use any google user to create service account, it doesn’t require service account owner is a user in G Suite. But G Suite administrator must authorize service account in G Suite Admin Console to access user mailbox.

  • G Suite Administrator should open admin.google.com, go to Admin Console, click "Security" > API Control;

    Authorize Service Account by G Suite Administrator
  • Click Add new and enter your service account client ID.

  • Enter the client ID of the service account or OAuth2 client ID of the app.

  • In OAuth Scopes, add each scope that the application can access (should be appropriately narrow). and input https://mail.google.com/, email, profile in One or More API Scopes, click "Authorize".

    manage api client access by G Suite Administrator 1

After G Suite administrator authorized service account, you can use it to access any users mailbox in G Suite domain.

Enable TLS Strong Encryption Algorithms in .NET 2.0 and .NET 4.0

Because HttpWebRequest is used to get access token from web service. If you’re using legacy .NET framework (.NET 2.0 - .NET 3.5 and .NET 4.0 - 4.6.1), you need to enable Strong Encryption Algorithms to request access token:

Put the following content to a file named NetStrongEncrypt.reg, right-click this file -> Merge -> Yes. You can also download it from https://www.emailarchitect.net/webapp/download/NetStrongEncrypt.zip.

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v2.0.50727]
"SystemDefaultTlsVersions"=dword:00000001
"SchUseStrongCrypto"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v2.0.50727]
"SystemDefaultTlsVersions"=dword:00000001
"SchUseStrongCrypto"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319]
"SystemDefaultTlsVersions"=dword:00000001
"SchUseStrongCrypto"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319]
"SystemDefaultTlsVersions"=dword:00000001
"SchUseStrongCrypto"=dword:00000001

Access token lifetime

You don’t have to request access token every time. By default, access token expiration time is 3600 seconds, you can reuse the access token repeatedly before it is expired.

Delphi - Retrieve email using Gmail/G Suite OAuth 2.0 from IMAP4 server with service account - example

program Project1;
{$APPTYPE CONSOLE}

uses
    Windows, Messages, SysUtils, Variants, Classes, Graphics, ActiveX, MSXML2_TLB, EAGetMailObjLib_TLB;

const
    MailServerPop3 = 0;
    MailServerImap4 = 1;
    MailServerEWS = 2;
    MailServerDAV = 3;
    MailServerMsGraph = 4;

    // Auth type
    MailServerAuthLogin = 0;
    MailServerAuthCRAM5 = 1;
    MailServerAuthNTLM = 2;
    MailServerAuthXOAUTH2 = 3;

    CRYPT_MACHINE_KEYSET = 32;
    CRYPT_USER_KEYSET = 4096;
    CERT_SYSTEM_STORE_CURRENT_USER = 65536;
    CERT_SYSTEM_STORE_LOCAL_MACHINE = 131072;

    // GetMailInfosParam Flags
    GetMailInfos_All = 1;
    GetMailInfos_NewOnly = 2;
    GetMailInfos_ReadOnly = 4;
    GetMailInfos_SeqRange = 8;
    GetMailInfos_UIDRange = 16;
    GetMailInfos_PR_ENTRYID = 32;
    GetMailInfos_DateRange = 64;
    GetMailInfos_OrderByDateTime = 128;

function RequestAccessToken(requestData: WideString): WideString;
var
    httpRequest: TServerXMLHTTP;
    oauthParser: TOAuthResponseParser;
    fullRequest: OleVariant;
    status: integer;
    responseText: WideString;
    accessToken: WideString;
begin
    result := '';
    httpRequest := TServerXMLHTTP.Create(nil);

    fullRequest := 'grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=';
    fullRequest := fullRequest + requestData;

    httpRequest.setOption(2, 13056);
    httpRequest.open('POST', 'https://oauth2.googleapis.com/token', true);
    httpRequest.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
    httpRequest.send(fullRequest);

    while( httpRequest.readyState <> 4 ) do
    begin
        try
            httpRequest.waitForResponse(1);

        except
            writeln('Server response timeout (access token).');
            exit;
        end;
    end;

    status := httpRequest.status;
    responseText := httpRequest.responseText;

    if (status < 200) or (status >= 300) then
    begin
        writeln('Failed to get access token from server.' + responseText);
        exit;
    end;

    oauthParser := TOAuthResponseParser.Create(nil);
    oauthParser.Load(responseText);

    accessToken := oauthParser.AccessToken;

    if accessToken = '' then
    begin
        writeln('Failed to parse access token from server response.');
        exit;
    end;

    result := accessToken;
end;

function GenerateRequestData(gsuiteUser: WideString): WideString;
const
    // service account email address
    serviceAccount: WideString = 'xxxx@xxxx.iam.gserviceaccount.com';
    scope: WideString = 'https://mail.google.com/';
    aud: WideString = 'https://oauth2.googleapis.com/token';
var
    jwt: TSimpleJsonParser;
    cert: TCertificate;
    pfxPath: WideString;
    header: WideString;
    playload: WideString;
    signature: WideString;
    iat, exp: integer;
begin
    result := '';

    SetThreadLocale(GetSystemDefaultLCID());

    jwt := TSimpleJsonParser.Create(nil);

    header := jwt.JwtBase64UrlEncode('{"alg":"RS256","typ":"JWT"}');

    // token request timestamp
    iat := jwt.GetCurrentIAT();
    // token expiration time
    exp := iat + 3600;

    playload := '{';
    playload := playload + '"iss":"' + serviceAccount + '",';
    playload := playload + '"scope":"' + scope + '",';
    playload := playload + '"aud":"' + aud + '",';
    playload := playload + '"exp":' + IntToStr(exp) + ',';
    playload := playload + '"iat":' + IntToStr(iat) + ',';
    playload := playload + '"sub":"' + gsuiteUser + '"';
    playload := playload + '}';

    playload := jwt.JwtBase64UrlEncode(playload);

    cert := TCertificate.Create(nil);

    // load service account certificate to sign request data
    pfxPath := 'D:\MyData\GSuite\outh-77aec4d192ec.p12';
    cert.LoadFromFile(pfxPath, 'notasecret', CRYPT_USER_KEYSET);

    signature := jwt.SignRs256(cert.DefaultInterface, header + '.' + playload);
    if signature = '' then
    begin
        writeln('Failed to sign request data!');
        exit;
    end;

    result := header + '.' + playload + '.' + signature;
end;

procedure RetrieveEmail();
var
    gsuiteUser, accessToken: WideString;
    oServer: TMailServer;
    oClient: TMailClient;
    oTools: TTools;
    oMail: IMail;
    infos: IMailInfoCollection;
    oInfo: IMailInfo;
    localInbox, fileName: WideString;
    i: Integer;
begin

    try
        gsuiteUser := 'user@gsuitedomain';

        // request access token with service account
        // gsuiteUser is the full email address of the user in GSuite, for example: user@gsuitedomain
        accessToken := RequestAccessToken(GenerateRequestData(gsuiteUser));
        if accessToken = '' then
            exit;

        // set current thread code page to system default code page.
        SetThreadLocale(GetSystemDefaultLCID());
        oTools := TTools.Create(nil);

        // Create a folder named "inbox" under
        // current directory to store the email files
        localInbox := GetCurrentDir() + '\inbox';
        oTools.CreateFolder(localInbox);

        oServer := TMailServer.Create(nil);
        // Gmail IMAP Server
        oServer.Server := 'imap.gmail.com';
        // Use OAUTH 2.0
        oServer.AuthType := MailServerAuthXOAUTH2;

        oServer.User := gsuiteUser;
        // Use access token as password
        oServer.Password := accesstoken;
        // Use IMAP Protocol
        oServer.Protocol := MailServerImap4;

        // Enable SSL Connection
        oServer.SSLConnection := true;
        // Set IMAP SSL Port
        oServer.Port := 993;

        oClient := TMailClient.Create(nil);
        oClient.LicenseCode := 'TryIt';

        writeln('Connecting ' + oServer.Server + ' ...');
        oClient.Connect1(oServer.DefaultInterface);
        writeln('Connected!');

        // Get new email only, if you want to get all emails, please remove this line
        oClient.GetMailInfosParam.GetMailInfosOptions := GetMailInfos_NewOnly;

        infos := oClient.GetMailInfoList();
        writeln(Format('Total %d email(s)', [infos.Count]));

        for i := 0 to infos.Count - 1 do
        begin
            oInfo := infos.Item[i];

            writeln(Format('Index: %d; Size: %d; UIDL: ' + oInfo.UIDL,
            [oInfo.Index, oInfo.Size]));

            // Generate a random file name by current local datetime,
            // You can use your method to generate the filename if you do not like it
            fileName := localInbox + '\' + oTools.GenFileName(i) + '.eml';

            // Receive email from IMAP server
            oMail := oClient.GetMail(oInfo);

            writeln('From: ' + oMail.From.Address + #13#10 +
                'Subject: ' + oMail.Subject);

            // Save email to local disk
            oMail.SaveAs(fileName, true);

            // Mark email as read to prevent retrieving this email again.
            oClient.MarkAsRead(oInfo, true);

            // If you want to delete current email, please use Delete method instead of MarkAsRead
            // oClient.Delete(oInfo);
        end;

        // Quit and expunge emails marked as deleted from IMAP server
        oClient.Quit;
    except
        on ep:Exception do
            writeln('Error: ' + ep.Message);
        end;

end;

begin

    CoInitialize(nil);

    writeln('+------------------------------------------------------------------+');
    writeln('  Sign in with Google OAuth');
    writeln('   If you got "This app is not verified" information in Web Browser, ');
    writeln('      click "Advanced" -> Go to ... to continue test.');
    writeln('+------------------------------------------------------------------+');
    writeln('');

    writeln('Press ENTER key to sign in...');
    readln;

    RetrieveEmail();

    writeln('Press ENTER key to quit...');
    readln;

end.

TLS 1.2 protocol

TLS is the successor of SSL, more and more SMTP servers require TLS 1.2 encryption now.

If your operating system is Windows XP/Vista/Windows 7/Windows 2003/2008/2008 R2/2012/2012 R2, you need to enable TLS 1.2 protocol in your operating system like this:

Enable TLS 1.2 on Windows XP/Vista/7/10/Windows 2008/2008 R2/2012

Appendix

Comments

If you have any comments or questions about above example codes, please click here to add your comments.