Tracing connection between TDW_NOCOMPRESS SAPGUI environment variable to bothering window and actual data compression routine.

Dennis Yurichev <dennis@conus.info>

It is known that network traffic between SAPGUI and SAP is not crypted by default, it's actually compressed (read here and here). It is also known that by setting environment variable TDW_NOCOMPRESS to 1, it is possible to turn compression off.

You will see a bothering windows that cannot be closed:

screenshot

Let's see, if we can remove that window somehow.

But before this, let's see what we already know. First: we know that environment variable TDW_NOCOMPRESS is checked somewhere in SAPGUI client. Second: string like "data compression switched off" must be present somewhere too. With the help of FAR file manager I found that both of these strings are stored in the SAPguilib.dll file.

So let's open SAPguilib.dll in IDA and search for "TDW_NOCOMPRESS" string. Yes, it is present and there is only one reference to it.

We see the following piece of code (all file offsets are for SAPGUI 720 win32, SAPguilib.dll file version 7200,1,0,9009):

.text:6440D51B                 lea     eax, [ebp+2108h+var_211C]
.text:6440D51E                 push    eax             ; int
.text:6440D51F                 push    offset aTdw_nocompress ; "TDW_NOCOMPRESS"
.text:6440D524                 mov     byte ptr [edi+15h], 0
.text:6440D528                 call    chk_env
.text:6440D52D                 pop     ecx
.text:6440D52E                 pop     ecx
.text:6440D52F                 push    offset byte_64443AF8
.text:6440D534                 lea     ecx, [ebp+2108h+var_211C]
.text:6440D537                 call    ds:mfc90_1603   ; ?Compare@?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QBEHPBD@Z @ 1603 NONAME
.text:6440D53D                 test    eax, eax
.text:6440D53F                 jz      short loc_6440D55A
.text:6440D541                 lea     ecx, [ebp+2108h+var_211C]
.text:6440D544                 call    ds:mfc90_910    ; ??B?$CSimpleStringT@D$00@ATL@@QBEPBDXZ @ 910 NONAME
.text:6440D54A                 push    eax             ; Str
.text:6440D54B                 call    ds:atoi
.text:6440D551                 test    eax, eax
.text:6440D553                 setnz   al
.text:6440D556                 pop     ecx
.text:6440D557                 mov     [edi+15h], al

String returned by chk_env() via second argument is then handled by MFC string functions and then atoi() is called. After that, numerical value is stored to edi+15h.

Also, take a look onto chk_env() function (I gave this name to it):

.text:64413F20 ; int __cdecl chk_env(char *VarName, int)
.text:64413F20 chk_env         proc near               ; CODE XREF: load_command_line+A9.p
.text:64413F20                                         ; load_command_line+1A3.p ...
.text:64413F20
.text:64413F20 DstSize         = dword ptr -0Ch
.text:64413F20 var_8           = dword ptr -8
.text:64413F20 DstBuf          = dword ptr -4
.text:64413F20 VarName         = dword ptr  8
.text:64413F20 arg_4           = dword ptr  0Ch
.text:64413F20
.text:64413F20                 push    ebp
.text:64413F21                 mov     ebp, esp
.text:64413F23                 sub     esp, 0Ch
.text:64413F26                 mov     [ebp+DstSize], 0
.text:64413F2D                 mov     [ebp+DstBuf], 0
.text:64413F34                 push    offset unk_6444C88C
.text:64413F39                 mov     ecx, [ebp+arg_4]
.text:64413F3C                 call    ds:mfc90_820    ; ??4?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 820 NONAME
.text:64413F42                 mov     eax, [ebp+VarName]
.text:64413F45                 push    eax             ; VarName
.text:64413F46                 mov     ecx, [ebp+DstSize]
.text:64413F49                 push    ecx             ; DstSize
.text:64413F4A                 mov     edx, [ebp+DstBuf]
.text:64413F4D                 push    edx             ; DstBuf
.text:64413F4E                 lea     eax, [ebp+DstSize]
.text:64413F51                 push    eax             ; ReturnSize
.text:64413F52                 call    ds:getenv_s
.text:64413F58                 add     esp, 10h
.text:64413F5B                 mov     [ebp+var_8], eax
.text:64413F5E                 cmp     [ebp+var_8], 0
.text:64413F62                 jz      short loc_64413F68
.text:64413F64                 xor     eax, eax
.text:64413F66                 jmp     short loc_64413FBC
.text:64413F68 ; ---------------------------------------------------------------------------
.text:64413F68
.text:64413F68 loc_64413F68:                           ; CODE XREF: chk_env+42.j
.text:64413F68                 cmp     [ebp+DstSize], 0
.text:64413F6C                 jnz     short loc_64413F72
.text:64413F6E                 xor     eax, eax
.text:64413F70                 jmp     short loc_64413FBC
.text:64413F72 ; ---------------------------------------------------------------------------
.text:64413F72
.text:64413F72 loc_64413F72:                           ; CODE XREF: chk_env+4C.j
.text:64413F72                 mov     ecx, [ebp+DstSize]
.text:64413F75                 push    ecx
.text:64413F76                 mov     ecx, [ebp+arg_4]
.text:64413F79                 call    ds:mfc90_2691   ; ?GetBuffer@?$CSimpleStringT@D$00@ATL@@QAEPADH@Z @ 2691 NONAME
.text:64413F7F                 mov     [ebp+DstBuf], eax
.text:64413F82                 mov     edx, [ebp+VarName]
.text:64413F85                 push    edx             ; VarName
.text:64413F86                 mov     eax, [ebp+DstSize]
.text:64413F89                 push    eax             ; DstSize
.text:64413F8A                 mov     ecx, [ebp+DstBuf]
.text:64413F8D                 push    ecx             ; DstBuf
.text:64413F8E                 lea     edx, [ebp+DstSize]
.text:64413F91                 push    edx             ; ReturnSize
.text:64413F92                 call    ds:getenv_s
.text:64413F98                 add     esp, 10h
.text:64413F9B                 mov     [ebp+var_8], eax
.text:64413F9E                 push    0FFFFFFFFh
.text:64413FA0                 mov     ecx, [ebp+arg_4]
.text:64413FA3                 call    ds:mfc90_5835   ; ?ReleaseBuffer@?$CSimpleStringT@D$00@ATL@@QAEXH@Z @ 5835 NONAME
.text:64413FA9                 cmp     [ebp+var_8], 0
.text:64413FAD                 jz      short loc_64413FB3
.text:64413FAF                 xor     eax, eax
.text:64413FB1                 jmp     short loc_64413FBC
.text:64413FB3 ; ---------------------------------------------------------------------------
.text:64413FB3
.text:64413FB3 loc_64413FB3:                           ; CODE XREF: chk_env+8D.j
.text:64413FB3                 mov     ecx, [ebp+arg_4]
.text:64413FB6                 call    ds:mfc90_910    ; ??B?$CSimpleStringT@D$00@ATL@@QBEPBDXZ @ 910 NONAME
.text:64413FBC
.text:64413FBC loc_64413FBC:                           ; CODE XREF: chk_env+46.j
.text:64413FBC                                         ; chk_env+50.j ...
.text:64413FBC                 mov     esp, ebp
.text:64413FBE                 pop     ebp
.text:64413FBF                 retn
.text:64413FBF chk_env         endp

Yes. getenv_s() function is Microsoft security-enhanced version of getenv().

There are also some MFC string manipulations.

Lots of other environment variables are checked as well. Here is a list of all variables being checked and what SAPGUI could write to trace log when logging is turned on:

"DPTRACE""GUI-OPTION: Trace set to %d"
"TDW_HEXDUMP""GUI-OPTION: Hexdump enabled"
"TDW_WORKDIR""GUI-OPTION: working directory '%s'"
"TDW_SPLASHSRCEENOFF""GUI-OPTION: Splash Screen Off" / "GUI-OPTION: Splash Screen On"
"TDW_REPLYTIMEOUT""GUI-OPTION: reply timeout %d milliseconds"
"TDW_PLAYBACKTIMEOUT""GUI-OPTION: PlaybackTimeout set to %d milliseconds"
"TDW_NOCOMPRESS""GUI-OPTION: no compression read"
"TDW_EXPERT""GUI-OPTION: expert mode"
"TDW_PLAYBACKPROGRESS""GUI-OPTION: PlaybackProgress"
"TDW_PLAYBACKNETTRAFFIC""GUI-OPTION: PlaybackNetTraffic"
"TDW_PLAYLOG""GUI-OPTION: /PlayLog is YES, file %s"
"TDW_PLAYTIME""GUI-OPTION: /PlayTime set to %d milliseconds"
"TDW_LOGFILE""GUI-OPTION: TDW_LOGFILE '%s'"
"TDW_WAN""GUI-OPTION: WAN - low speed connection enabled"
"TDW_FULLMENU""GUI-OPTION: FullMenu enabled"
"SAP_CP" / "SAP_CODEPAGE""GUI-OPTION: SAP_CODEPAGE '%d'"
"UPDOWNLOAD_CP""GUI-OPTION: UPDOWNLOAD_CP '%d'"
"SNC_PARTNERNAME""GUI-OPTION: SNC name '%s'"
"SNC_QOP""GUI-OPTION: SNC_QOP '%s'"
"SNC_LIB""GUI-OPTION: SNC is set to: %s"
"SAPGUI_INPLACE""GUI-OPTION: environment variable SAPGUI_INPLACE is on"

It can be seen in the function, that each variable can be also set by specifying (slightly different options) in command line.

Settings for each variable are written to the array via pointer in EDI register. EDI is being set before that function call:

.text:6440EE00                 lea     edi, [ebp+2884h+var_2884] ; options here like +0x15...
.text:6440EE03                 lea     ecx, [esi+24h]
.text:6440EE06                 call    load_command_line
.text:6440EE0B                 mov     edi, eax
.text:6440EE0D                 xor     ebx, ebx
.text:6440EE0F                 cmp     edi, ebx
.text:6440EE11                 jz      short loc_6440EE42
.text:6440EE13                 push    edi
.text:6440EE14                 push    offset aSapguiStoppedA ; "Sapgui stopped after commandline interp"...
.text:6440EE19                 push    dword_644F93E8
.text:6440EE1F                 call    FEWTraceError

Now, can we find "data record mode switched on" string? Yes, and here is the only reference in function CDwsGui::PrepareInfoWindow(). How did I got class/method names? There is a lot of special debugging calls like:

.text:64405160                 push    dword ptr [esi+2854h]
.text:64405166                 push    offset aCdwsguiPrepare ; "\nCDwsGui::PrepareInfoWindow: sapgui env"...
.text:6440516B                 push    dword ptr [esi+2848h]
.text:64405171                 call    dbg
.text:64405176                 add     esp, 0Ch

... or:

.text:6440237A                 push    eax
.text:6440237B                 push    offset aCclientStart_6 ; "CClient::Start: set shortcut user to '%"...
.text:64402380                 push    dword ptr [edi+4]
.text:64402383                 call    dbg
.text:64402388                 add     esp, 0Ch

It is *very* useful.

So let's see contents of that "bothering window" function:

.text:64404F4F CDwsGui__PrepareInfoWindow proc near    ; CODE XREF: CDwsGui__OnReceiveMessage-142.p
.text:64404F4F
.text:64404F4F pvParam         = byte ptr -3Ch
.text:64404F4F var_38          = dword ptr -38h
.text:64404F4F var_34          = dword ptr -34h
.text:64404F4F rc              = tagRECT ptr -2Ch
.text:64404F4F cy              = dword ptr -1Ch
.text:64404F4F h               = dword ptr -18h
.text:64404F4F var_14          = dword ptr -14h
.text:64404F4F var_10          = dword ptr -10h
.text:64404F4F var_4           = dword ptr -4
.text:64404F4F
.text:64404F4F                 push    30h
.text:64404F51                 mov     eax, offset loc_64438E00
.text:64404F56                 call    __EH_prolog3
.text:64404F5B                 mov     esi, ecx        ; ECX is pointer to object
.text:64404F5D                 xor     ebx, ebx
.text:64404F5F                 lea     ecx, [ebp+var_14]
.text:64404F62                 mov     [ebp+var_10], ebx
.text:64404F65                 call    ds:mfc90_316    ; ??0?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAE@XZ @ 316 NONAME
.text:64404F6B                 mov     [ebp+var_4], ebx
.text:64404F6E                 lea     edi, [esi+2854h]
.text:64404F74                 push    offset aEnvironmentInf ; "Environment information:\n"
.text:64404F79                 mov     ecx, edi
.text:64404F7B                 call    ds:mfc90_820    ; ??4?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 820 NONAME
.text:64404F81                 cmp     [esi+38h], ebx
.text:64404F84                 mov     ebx, ds:mfc90_2539 ; ?Format@?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAAXPBDZZ @ 2539 NONAME
.text:64404F8A                 jbe     short loc_64404FA9
.text:64404F8C                 push    dword ptr [esi+34h]
.text:64404F8F                 lea     eax, [ebp+var_14]
.text:64404F92                 push    offset aWorkingDirecto ; "working directory: '%s'\n"
.text:64404F97                 push    eax
.text:64404F98                 call    ebx ; mfc90_2539 ; ?Format@?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAAXPBDZZ @ 2539 NONAME
.text:64404F9A                 add     esp, 0Ch
.text:64404F9D                 lea     eax, [ebp+var_14]
.text:64404FA0                 push    eax
.text:64404FA1                 mov     ecx, edi
.text:64404FA3                 call    ds:mfc90_941    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@ABV?$CSimpleStringT@D$00@1@@Z @ 941 NONAME
.text:64404FA9
.text:64404FA9 loc_64404FA9:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+3B.j
.text:64404FA9                 mov     eax, [esi+38h]
.text:64404FAC                 test    eax, eax
.text:64404FAE                 jbe     short loc_64404FD3
.text:64404FB0                 push    eax
.text:64404FB1                 lea     eax, [ebp+var_14]
.text:64404FB4                 push    offset aTraceLevelDAct ; "trace level %d activated\n"
.text:64404FB9                 push    eax
.text:64404FBA                 call    ebx ; mfc90_2539 ; ?Format@?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAAXPBDZZ @ 2539 NONAME
.text:64404FBC                 add     esp, 0Ch
.text:64404FBF                 lea     eax, [ebp+var_14]
.text:64404FC2                 push    eax
.text:64404FC3                 mov     ecx, edi
.text:64404FC5                 call    ds:mfc90_941    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@ABV?$CSimpleStringT@D$00@1@@Z @ 941 NONAME
.text:64404FCB                 xor     ebx, ebx
.text:64404FCD                 inc     ebx
.text:64404FCE                 mov     [ebp+var_10], ebx
.text:64404FD1                 jmp     short loc_64404FD6
.text:64404FD3 ; ---------------------------------------------------------------------------
.text:64404FD3
.text:64404FD3 loc_64404FD3:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+5F.j
.text:64404FD3                 xor     ebx, ebx
.text:64404FD5                 inc     ebx
.text:64404FD6
.text:64404FD6 loc_64404FD6:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+82.j
.text:64404FD6                 cmp     [esi+38h], ebx
.text:64404FD9                 jbe     short loc_64404FF1
.text:64404FDB                 cmp     dword ptr [esi+2978h], 0
.text:64404FE2                 jz      short loc_64404FF1
.text:64404FE4                 push    offset aHexdumpInTrace ; "hexdump in trace activated\n"
.text:64404FE9                 mov     ecx, edi
.text:64404FEB                 call    ds:mfc90_945    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 945 NONAME
.text:64404FF1
.text:64404FF1 loc_64404FF1:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+8A.j
.text:64404FF1                                         ; CDwsGui__PrepareInfoWindow+93.j
.text:64404FF1                 cmp     byte ptr [esi+78h], 0
.text:64404FF5                 jz      short loc_64405007
.text:64404FF7                 push    offset aLoggingActivat ; "logging activated\n"
.text:64404FFC                 mov     ecx, edi
.text:64404FFE                 call    ds:mfc90_945    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 945 NONAME
.text:64405004                 mov     [ebp+var_10], ebx
.text:64405007
.text:64405007 loc_64405007:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+A6.j
.text:64405007                 cmp     byte ptr [esi+3Dh], 0
.text:6440500B                 jz      short bypass
.text:6440500D                 push    offset aDataCompressio ; "data compression switched off\n"
.text:64405012                 mov     ecx, edi
.text:64405014                 call    ds:mfc90_945    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 945 NONAME
.text:6440501A                 mov     [ebp+var_10], ebx
.text:6440501D
.text:6440501D bypass:                                 ; CODE XREF: CDwsGui__PrepareInfoWindow+BC.j
.text:6440501D                 mov     eax, [esi+20h]
.text:64405020                 test    eax, eax
.text:64405022                 jz      short loc_6440503A
.text:64405024                 cmp     dword ptr [eax+28h], 0
.text:64405028                 jz      short loc_6440503A
.text:6440502A                 push    offset aDataRecordMode ; "data record mode switched on\n"
.text:6440502F                 mov     ecx, edi
.text:64405031                 call    ds:mfc90_945    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 945 NONAME
.text:64405037                 mov     [ebp+var_10], ebx
.text:6440503A
.text:6440503A loc_6440503A:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+D3.j
.text:6440503A                                         ; CDwsGui__PrepareInfoWindow+D9.j
.text:6440503A                 mov     ecx, edi
.text:6440503C                 cmp     [ebp+var_10], ebx
.text:6440503F                 jnz     loc_64405142
.text:64405045                 push    offset aForMaximumData ; "\nFor maximum data security delete\nthe s"...
.text:6440504A                 call    ds:mfc90_945    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 945 NONAME
.text:64405050                 xor     edi, edi
.text:64405052                 push    edi             ; fWinIni
.text:64405053                 lea     eax, [ebp+pvParam]
.text:64405056                 push    eax             ; pvParam
.text:64405057                 push    edi             ; uiParam
.text:64405058                 push    30h             ; uiAction
.text:6440505A                 call    ds:SystemParametersInfoA
.text:64405060                 mov     eax, [ebp+var_34]
.text:64405063                 cmp     eax, 1600
.text:64405068                 jle     short loc_64405072
.text:6440506A                 cdq
.text:6440506B                 sub     eax, edx
.text:6440506D                 sar     eax, 1
.text:6440506F                 mov     [ebp+var_34], eax
.text:64405072
.text:64405072 loc_64405072:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+119.j
.text:64405072                 push    edi             ; hWnd
.text:64405073                 mov     [ebp+cy], 0A0h
.text:6440507A                 call    ds:GetDC
.text:64405080                 mov     [ebp+var_10], eax
.text:64405083                 mov     ebx, 12Ch
.text:64405088                 cmp     eax, edi
.text:6440508A                 jz      loc_64405113
.text:64405090                 push    11h             ; i
.text:64405092                 call    ds:GetStockObject
.text:64405098                 mov     edi, ds:SelectObject
.text:6440509E                 push    eax             ; h
.text:6440509F                 push    [ebp+var_10]    ; hdc
.text:644050A2                 call    edi ; SelectObject
.text:644050A4                 and     [ebp+rc.left], 0
.text:644050A8                 and     [ebp+rc.top], 0
.text:644050AC                 mov     [ebp+h], eax
.text:644050AF                 push    401h            ; format
.text:644050B4                 lea     eax, [ebp+rc]
.text:644050B7                 push    eax             ; lprc
.text:644050B8                 lea     ecx, [esi+2854h]
.text:644050BE                 mov     [ebp+rc.right], ebx
.text:644050C1                 mov     [ebp+rc.bottom], 0B4h
.text:644050C8                 call    ds:mfc90_3178   ; ?GetLength@?$CSimpleStringT@D$00@ATL@@QBEHXZ @ 3178 NONAME
.text:644050CE                 push    eax             ; cchText
.text:644050CF                 lea     ecx, [esi+2854h]
.text:644050D5                 call    ds:mfc90_910    ; ??B?$CSimpleStringT@D$00@ATL@@QBEPBDXZ @ 910 NONAME
.text:644050DB                 push    eax             ; lpchText
.text:644050DC                 push    [ebp+var_10]    ; hdc
.text:644050DF                 call    ds:DrawTextA
.text:644050E5                 push    4               ; nIndex
.text:644050E7                 call    ds:GetSystemMetrics
.text:644050ED                 mov     ecx, [ebp+rc.bottom]
.text:644050F0                 sub     ecx, [ebp+rc.top]
.text:644050F3                 cmp     [ebp+h], 0
.text:644050F7                 lea     eax, [eax+ecx+28h]
.text:644050FB                 mov     [ebp+cy], eax
.text:644050FE                 jz      short loc_64405108
.text:64405100                 push    [ebp+h]         ; h
.text:64405103                 push    [ebp+var_10]    ; hdc
.text:64405106                 call    edi ; SelectObject
.text:64405108
.text:64405108 loc_64405108:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+1AF.j
.text:64405108                 push    [ebp+var_10]    ; hDC
.text:6440510B                 push    0               ; hWnd
.text:6440510D                 call    ds:ReleaseDC
.text:64405113
.text:64405113 loc_64405113:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+13B.j
.text:64405113                 mov     eax, [ebp+var_38]
.text:64405116                 push    80h             ; uFlags
.text:6440511B                 push    [ebp+cy]        ; cy
.text:6440511E                 inc     eax
.text:6440511F                 push    ebx             ; cx
.text:64405120                 push    eax             ; Y
.text:64405121                 mov     eax, [ebp+var_34]
.text:64405124                 add     eax, 0FFFFFED4h
.text:64405129                 cdq
.text:6440512A                 sub     eax, edx
.text:6440512C                 sar     eax, 1
.text:6440512E                 push    eax             ; X
.text:6440512F                 push    0               ; hWndInsertAfter
.text:64405131                 push    dword ptr [esi+285Ch] ; hWnd
.text:64405137                 call    ds:SetWindowPos
.text:6440513D                 xor     ebx, ebx
.text:6440513F                 inc     ebx
.text:64405140                 jmp     short loc_6440514D
.text:64405142 ; ---------------------------------------------------------------------------
.text:64405142
.text:64405142 loc_64405142:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+F0.j
.text:64405142                 push    offset byte_64443AF8
.text:64405147                 call    ds:mfc90_820    ; ??4?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 820 NONAME
.text:6440514D
.text:6440514D loc_6440514D:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+1F1.j
.text:6440514D                 cmp     dword_6450B970, ebx
.text:64405153                 jl      short loc_64405188
.text:64405155                 call    sub_6441C910
.text:6440515A                 mov     dword_644F858C, ebx
.text:64405160                 push    dword ptr [esi+2854h]
.text:64405166                 push    offset aCdwsguiPrepare ; "\nCDwsGui::PrepareInfoWindow: sapgui env"...
.text:6440516B                 push    dword ptr [esi+2848h]
.text:64405171                 call    dbg
.text:64405176                 add     esp, 0Ch
.text:64405179                 mov     dword_644F858C, 2
.text:64405183                 call    sub_6441C920
.text:64405188
.text:64405188 loc_64405188:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+204.j
.text:64405188                 or      [ebp+var_4], 0FFFFFFFFh
.text:6440518C                 lea     ecx, [ebp+var_14]
.text:6440518F                 call    ds:mfc90_601    ; ??1?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAE@XZ @ 601 NONAME
.text:64405195                 call    __EH_epilog3
.text:6440519A                 retn
.text:6440519A CDwsGui__PrepareInfoWindow endp

ECX at function start puts pointer to object. In our case, that object obviously has class type CDwsGui. If some option variables in that object are different, specific message part will be concatenated to resulting message.

If value at +0x3D offset is not zero, compression is off:

.text:64405007 loc_64405007:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+A6.j
.text:64405007                 cmp     byte ptr [esi+3Dh], 0
.text:6440500B                 jz      short bypass
.text:6440500D                 push    offset aDataCompressio ; "data compression switched off\n"
.text:64405012                 mov     ecx, edi
.text:64405014                 call    ds:mfc90_945    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 945 NONAME
.text:6440501A                 mov     [ebp+var_10], ebx
.text:6440501D
.text:6440501D bypass:                                 ; CODE XREF: CDwsGui__PrepareInfoWindow+BC.j

It is interesting, that finally, var_10 state defines whether the message is to be shown or not at all:

.text:6440503C                 cmp     [ebp+var_10], ebx
.text:6440503F                 jnz     exit ; bypass drawing

; add strings "For maximum data security delete" / "the setting(s) as soon as possible !":

.text:64405045                 push    offset aForMaximumData ; "\nFor maximum data security delete\nthe s"...
.text:6440504A                 call    ds:mfc90_945    ; ??Y?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@PBD@Z @ 945 NONAME
.text:64405050                 xor     edi, edi
.text:64405052                 push    edi             ; fWinIni
.text:64405053                 lea     eax, [ebp+pvParam]
.text:64405056                 push    eax             ; pvParam
.text:64405057                 push    edi             ; uiParam
.text:64405058                 push    30h             ; uiAction
.text:6440505A                 call    ds:SystemParametersInfoA
.text:64405060                 mov     eax, [ebp+var_34]
.text:64405063                 cmp     eax, 1600
.text:64405068                 jle     short loc_64405072
.text:6440506A                 cdq
.text:6440506B                 sub     eax, edx
.text:6440506D                 sar     eax, 1
.text:6440506F                 mov     [ebp+var_34], eax
.text:64405072
.text:64405072 loc_64405072:                           ; CODE XREF: CDwsGui__PrepareInfoWindow+119.j

start drawing:

.text:64405072                 push    edi             ; hWnd
.text:64405073                 mov     [ebp+cy], 0A0h
.text:6440507A                 call    ds:GetDC

Let's check our theory on practice.

JNZ at this line ...

.text:6440503F                 jnz     exit ; bypass drawing
... replace it with just JMP, and get SAPGUI working without that bothering window appearing!

Now let's dig deeper and find connection between 0x15 offset in load_command_line() (I gave that name to that function) and 0x3D offset in CDwsGui::PrepareInfoWindow. Are we sure that the value is the same?

I'm starting to search for all occurences of 0x15 value in code. For some not very large programs like SAPGUI, it sometimes works. Here is the first occurence I got:

.text:64404C19 sub_64404C19    proc near               ; CODE XREF: CDwsGui__CopyOptions+39.p
.text:64404C19
.text:64404C19 arg_0           = dword ptr  4
.text:64404C19
.text:64404C19                 push    ebx
.text:64404C1A                 push    ebp
.text:64404C1B                 push    esi
.text:64404C1C                 push    edi
.text:64404C1D                 mov     edi, [esp+10h+arg_0]
.text:64404C21                 mov     eax, [edi]
.text:64404C23                 mov     esi, ecx ; ESI/ECX are pointers to some unknown object.
.text:64404C25                 mov     [esi], eax
.text:64404C27                 mov     eax, [edi+4]
.text:64404C2A                 mov     [esi+4], eax
.text:64404C2D                 mov     eax, [edi+8]
.text:64404C30                 mov     [esi+8], eax
.text:64404C33                 lea     eax, [edi+0Ch]
.text:64404C36                 push    eax
.text:64404C37                 lea     ecx, [esi+0Ch]
.text:64404C3A                 call    ds:mfc90_817    ; ??4?$CStringT@DV?$StrTraitMFC_DLL@DV?$ChTraitsCRT@D@ATL@@@@@ATL@@QAEAAV01@ABV01@@Z @ 817 NONAME
.text:64404C40                 mov     eax, [edi+10h]
.text:64404C43                 mov     [esi+10h], eax
.text:64404C46                 mov     al, [edi+14h]
.text:64404C49                 mov     [esi+14h], al
.text:64404C4C                 mov     al, [edi+15h] ; copy byte from 0x15 offset
.text:64404C4F                 mov     [esi+15h], al ; to 0x15 offset in CDwsGui object

That function was called from the function CDwsGui::CopyOptions! And thanks again for debugging information.

But the real answer in the function CDwsGui::Init():

.text:6440B0BF loc_6440B0BF:                           ; CODE XREF: CDwsGui__Init+C.j
.text:6440B0BF                 mov     eax, [ebp+arg_0]
.text:6440B0C2                 push    [ebp+arg_4]
.text:6440B0C5                 mov     [esi+2844h], eax
.text:6440B0CB                 lea     eax, [esi+28h] ; ESI is pointer to CDwsGui object
.text:6440B0CE                 push    eax
.text:6440B0CF                 call    CDwsGui__CopyOptions

Finally, we know: array filled in load_command_line() is actually placed in CDwsGui object but on +0x28 offset. 0x15 + 0x28 is exactly 0x3D. OK, we found the place where the value is copied to.

Let's also find other places where "0x3D" offset is used. Here is one of them in CDwsGui::SapguiRun (again, thanks to debugging calls):

.text:64409D58                 cmp     [esi+3Dh], bl   ; ESI is pointer to CDwsGui object
.text:64409D5B                 lea     ecx, [esi+2B8h]
.text:64409D61                 setz    al
.text:64409D64                 push    eax             ; arg_10 of CConnectionContext::CreateNetwork
.text:64409D65                 push    dword ptr [esi+64h]
.text:64409D68                 call    ds:mfc90_910    ; ??B?$CSimpleStringT@D$00@ATL@@QBEPBDXZ @ 910 NONAME
.text:64409D68                                         ; no arguments
.text:64409D6E                 push    eax
.text:64409D6F                 lea     ecx, [esi+2BCh]
.text:64409D75                 call    ds:mfc90_910    ; ??B?$CSimpleStringT@D$00@ATL@@QBEPBDXZ @ 910 NONAME
.text:64409D75                                         ; no arguments
.text:64409D7B                 push    eax
.text:64409D7C                 push    esi
.text:64409D7D                 lea     ecx, [esi+8]
.text:64409D80                 call    CConnectionContext__CreateNetwork

Let's check our findings. Replace "setz al" here to "xor eax, eax / nop", clear TDW_NOCOMPRESS environment variable and run SAPGUI. Wow! There is no bothering window (just as expected, because variable is not set), but in Wireshark we can see that the packets are not compressed! Obviously, this is the place where compression flag is to be set in CConnectionContext object.

So, compression flag is the 5th argument of function CConnectionContext::CreateNetwork. Inside that function, another one is called:

...
.text:64403476                 push    [ebp+compression]
.text:64403479                 push    [ebp+arg_C]
.text:6440347C                 push    [ebp+arg_8]
.text:6440347F                 push    [ebp+arg_4]
.text:64403482                 push    [ebp+arg_0]
.text:64403485                 call    CNetwork__CNetwork

Compression flag is the 5th argument to CNetwork::CNetwork constructor.

And here is how CNetwork constructor sets some flag in CNetwork object according to the 5th argument *and* some another value which probably could affect network compression too.

.text:64411DF1                 cmp     [ebp+compression], esi
.text:64411DF7                 jz      short set_EAX_to_0
.text:64411DF9                 mov     al, [ebx+78h]   ; another value may affect compression?
.text:64411DFC                 cmp     al, '3'
.text:64411DFE                 jz      short set_EAX_to_1
.text:64411E00                 cmp     al, '4'
.text:64411E02                 jnz     short set_EAX_to_0
.text:64411E04
.text:64411E04 set_EAX_to_1:                           ; CODE XREF: CNetwork__CNetwork+7E9.j
.text:64411E04                 xor     eax, eax
.text:64411E06                 inc     eax             ; EAX -> 1
.text:64411E07                 jmp     short loc_64411E0B
.text:64411E09 ; ---------------------------------------------------------------------------
.text:64411E09
.text:64411E09 set_EAX_to_0:                           ; CODE XREF: CNetwork__CNetwork+7E2.j
.text:64411E09                                         ; CNetwork__CNetwork+7ED.j
.text:64411E09                 xor     eax, eax        ; EAX -> 0
.text:64411E0B
.text:64411E0B loc_64411E0B:                           ; CODE XREF: CNetwork__CNetwork+7F2.j
.text:64411E0B                 mov     [ebx+3A4h], eax ; EBX is pointer to CNetwork object

At this point we know that compression flag is stored in CNetwork object by offset +0x3A4

Now let's dig across SAPguilib.dll for 0x3A4 value. And here is the second occurence in CDwsGui::OnClientMessageWrite (endless thanks for debugging information):

.text:64406F76 loc_64406F76:                           ; CODE XREF: CDwsGui__OnClientMessageWrite+2B9.j
.text:64406F76                 mov     ecx, [ebp+7728h+var_7794]
.text:64406F79                 cmp     dword ptr [ecx+3A4h], 1
.text:64406F80                 jnz     compression_flag_is_zero
.text:64406F86                 mov     byte ptr [ebx+7], 1
.text:64406F8A                 mov     eax, [esi+18h]
.text:64406F8D                 mov     ecx, eax
.text:64406F8F                 test    eax, eax
.text:64406F91                 ja      short loc_64406FFF
.text:64406F93                 mov     ecx, [esi+14h]
.text:64406F96                 mov     eax, [esi+20h]
.text:64406F99
.text:64406F99 loc_64406F99:                           ; CODE XREF: CDwsGui__OnClientMessageWrite+34F.j
.text:64406F99                 push    dword ptr [edi+2868h] ; int
.text:64406F9F                 lea     edx, [ebp+7728h+var_77A4]
.text:64406FA2                 push    edx             ; int
.text:64406FA3                 push    30000           ; int
.text:64406FA8                 lea     edx, [ebp+7728h+Dst]
.text:64406FAB                 push    edx             ; Dst
.text:64406FAC                 push    ecx             ; int
.text:64406FAD                 push    eax             ; Src
.text:64406FAE                 push    dword ptr [edi+28C0h] ; int
.text:64406FB4                 call    sub_644055C5       ; actual compression routine
.text:64406FB9                 add     esp, 1Ch
.text:64406FBC                 cmp     eax, 0FFFFFFF6h
.text:64406FBF                 jz      short loc_64407004
.text:64406FC1                 cmp     eax, 1
.text:64406FC4                 jz      loc_6440708C
.text:64406FCA                 cmp     eax, 2
.text:64406FCD                 jz      short loc_64407004
.text:64406FCF                 push    eax
.text:64406FD0                 push    offset aCompressionErr ; "compression error [rc = %d]- program wi"...
.text:64406FD5                 push    offset aGui_err_compre ; "GUI_ERR_COMPRESS"
.text:64406FDA                 push    dword ptr [edi+28D0h]
.text:64406FE0                 call    SapPcTxtRead

Let's take a look into sub_644055C5. In it we can only see call to memcpy() and some other function named (by IDA) sub_64417440.

And, let's take a look inside sub_64417440. What we see is:

.text:6441747C                 push    offset aErrorCsrcompre ; "\nERROR: CsRCompress: invalid handle"
.text:64417481                 call    eax ; dword_644F94C8
.text:64417483                 add     esp, 4

Voila! We've found the function which actually compresses data. As I revealed in past, this function is used in SAP and also open-source MaxDB project. So it is available in sources.

Doing last check here:

.text:64406F79                 cmp     dword ptr [ecx+3A4h], 1
.text:64406F80                 jnz     compression_flag_is_zero

Replace JNZ here for unconditional JMP. Remove environment variable TDW_NOCOMPRESS. Voila! In Wireshark we see that client messages are not compressed. Server responses, however, are compressed.

So we found exact connection between environment variable and the point where data compression routine may be called or may be bypassed.

Need a reverse engineering like this? -> dennis@conus.info