Bug 92538

Created attachment 118980 [details] [review]
[1/4] Disable activation tests on Windows builds

These rely on the --systemd-activation code path, which is not
compiled for Windows.

Created attachment 118981 [details] [review]
[2/4] When running dbus-daemon --session in tests, override  listen address

Otherwise, we can't reliably run tests for Windows, because the default
listening address on Windows is "autolaunch:" which is global to
a machine, resulting in testing an installed dbus-daemon instead of
the one we intended to test.

---

Ralf, I think this is what caused the failure you describe on <https://bugs.freedesktop.org/show_bug.cgi?id=92298#c9>.

Created attachment 118982 [details] [review]
[3/4] When running TAP tests, filter out trailing \r from  Windows .exe

If we're running Windows executables using Wine, then tap-driver.sh
won't accept "1..4\r\n" as TAP syntax.

Created attachment 118983 [details] [review]
[4/4] refs test: reduce number of repeats under Wine

Under Wine, the API calls we use to do this are implemented via IPC
to wineserver, which makes it unreasonably slow to try to brute-force
bugs by having many threads stress-test refcounting. Do a few
repetitions just to verify that refcounting basically works, but
don't do the full stress-test.

I think 1/4 and 2/4 would affect genuine Windows; 4/4 is Wine-specific, and I'm not sure about 3/4.

There are some issues remaining, but I don't think any of them should necessarily block merging the rest to master: some tests working is better than no tests working!

I'm getting a failure in test-syslog, which I don't understand: when run with --tap, it exits 0 with no output, and without --tap, it prints "syslog: " and then exits. It looks as though OutputDebugStringA() is making it just exit immediately?

> ERROR: test-syslog
> ==================
> 
> # random seed: R02S2d69e7eb611f40932b98599ebca0af9a
> ERROR: test-syslog.exe - missing test plan

test-refs is still rather slow, and we should perhaps reduce the number of threads further. test-relay and test-dbus-daemon aren't fast either.

I also see failures in the "embedded tests" (test-dbus and test-bus) but I think we should definitely consider those to be a separate issue - they're a very different framework.

Created attachment 118989 [details] [review]
[2/4] When running dbus-daemon --session in tests, override listen address

---

Sorry, Attachment #118981 [details] was the wrong patch. Here's the right one.

Created attachment 118990 [details] [review]
[3/4] When running TAP tests, filter out trailing \r from Windows .exe

---

Attachment #118982 [details] was also not the intended patch.

Comment on attachment 118980 [details] [review]
[1/4] Disable activation tests on Windows builds

Review of attachment 118980 [details] [review]:
-----------------------------------------------------------------

looks good

(In reply to Ralf Habacker from comment #8)
> [1/4] Disable activation tests on Windows builds
...
> looks good

Thanks, applied. Fixed in git for 1.10.2 and 1.11.0. Any thoughts on the other three?

Comment on attachment 118989 [details] [review]
[2/4] When running dbus-daemon --session in tests, override listen address

Review of attachment 118989 [details] [review]:
-----------------------------------------------------------------

looks good

Comment on attachment 118983 [details] [review]
[4/4] refs test: reduce number of repeats under Wine

Review of attachment 118983 [details] [review]:
-----------------------------------------------------------------

::: test/internals/refs.c
@@ +208,5 @@
>  
> +  f->n_threads = N_THREADS;
> +  f->n_refs = N_REFS;
> +
> +  if (g_getenv ("WINESERVERSOCKET") != NULL)

I guess this environment variable is for detecting if dbus is running under wine. This can be automated by checking the presence of a dedicated registry key for example HKCU/Software/Wine/Debug http://wiki.winehq.org/UsefulRegistryKeys

(In reply to Ralf Habacker from comment #11)
> Comment on attachment 118983 [details] [review]
> [4/4] refs test: reduce number of repeats under Wine
...
> > +  if (g_getenv ("WINESERVERSOCKET") != NULL)
> 
> I guess this environment variable is for detecting if dbus is running under
> wine.

Yes.

From <http://wiki.winehq.org/DeveloperFaq#head-96551a33b168826cd805c2f62c4ea705ac02f350> it seems that the preferred way is to probe for wine_get_version() in ntdll; I'll see what that looks like as an alternative.

Explicitly probing for Wine does seem a bit dirty. One way to dodge this problem without that might be to set some reasonable timeout - maybe 2 or 3 seconds per test-case - and stop the test early if that timeout is reached. This test comes from Bug #39836 and was intended to catch Bug #38005, which is a thread-safety failure - when the bug was present, the test would only have failed probabilistically.

(In reply to Simon McVittie from comment #12)
> Explicitly probing for Wine does seem a bit dirty.

Detecting wine may be useful to dump this info with bug reports created with DBUS_VERBOSE. 

Also using the wine detection to set the limits looks more clean as refactoring the test case to timeout the loop because of the hidden timing problem on wine.

> This test comes from Bug #39836 and was intended to catch Bug #38005, which
> is a thread-safety failure - when the bug was present, the test would only
> have failed probabilistically.

I tried to run 

WINESERVERSOCKET=1  wine test/test-refs.exe

and got

wine client error:0: recvmsg: Socket operation on non-socket

Then I changed 

  if (g_getenv ("WINESERVERSOCKET") != NULL)

into

  if (TRUE)

and did run 

wine test/test-refs.exe

which return result as expected.

Comment on attachment 118990 [details] [review]
[3/4] When running TAP tests, filter out trailing \r from Windows .exe

Review of attachment 118990 [details] [review]:
-----------------------------------------------------------------

How to verify proper operation ? By running make check ?

(In reply to Ralf Habacker from comment #13)
> I tried to run 
> 
> WINESERVERSOCKET=1  wine test/test-refs.exe
> 
> and got
> 
> wine client error:0: recvmsg: Socket operation on non-socket

That isn't how it works. Running wine sets WINESERVERSOCKET for its child processes automatically (at least, it does here, with 32-bit Wine 1.6.2) - the wineserver is what links together all the running Wine executables' views of the emulated Windows system.

(In reply to Ralf Habacker from comment #14)
> How to verify proper operation ? By running make check ?

Yes, with Autotools, mingw-w64, and Wine binfmt support.

test-dbus and test-bus still fail for me (Comment #5), so ignore those if you are building --with-embedded-tests or --with-tests. The important thing is that the GLib tests (test-dbus-daemon, test-loopback, test-monitor etc.) should all pass in "make check".

Comment on attachment 118983 [details] [review]
[4/4] refs test: reduce number of repeats under Wine

Review of attachment 118983 [details] [review]:
-----------------------------------------------------------------

looks good

::: test/internals/refs.c
@@ +208,5 @@
>  
> +  f->n_threads = N_THREADS;
> +  f->n_refs = N_REFS;
> +
> +  if (g_getenv ("WINESERVERSOCKET") != NULL)

Using WINESERVERSOCKET looks good. Would be nice to have a note in the doc that WINESERVERSOCKET is set by wine for child processes.

Created attachment 119223 [details] [review]
Fix crash running test-syslog on windows/wine

Comment on attachment 119223 [details] [review]
Fix crash running test-syslog on windows/wine

Review of attachment 119223 [details] [review]:
-----------------------------------------------------------------

Ah, good catch. The old code had undefined behaviour.

Please merge for 1.10.

Comment on attachment 119223 [details] [review]
Fix crash running test-syslog on windows/wine

committed to dbus-1.10

Comment on attachment 118989 [details] [review]
[2/4] When running dbus-daemon --session in tests, override listen address

committed to dbus-1.10

Comment on attachment 118980 [details] [review]
[1/4] Disable activation tests on Windows builds

committed to dbus-1.10

(In reply to Simon McVittie from comment #15)
> (In reply to Ralf Habacker from comment #13)
> > I tried to run 
> > 
> > WINESERVERSOCKET=1  wine test/test-refs.exe
> > 
> > and got
> > 
> > wine client error:0: recvmsg: Socket operation on non-socket
> 
> That isn't how it works. Running wine sets WINESERVERSOCKET for its child
> processes automatically (at least, it does here, with 32-bit Wine 1.6.2) -
> the wineserver is what links together all the running Wine executables'
> views of the emulated Windows system.
> 
> (In reply to Ralf Habacker from comment #14)
> > How to verify proper operation ? By running make check ?
> 
> Yes, with Autotools, mingw-w64, and Wine binfmt support.
> 
> test-dbus and test-bus still fail for me (Comment #5), so ignore those if
> you are building --with-embedded-tests or --with-tests. The important thing
> is that the GLib tests (test-dbus-daemon, test-loopback, test-monitor etc.)
> should all pass in "make check".

sed -e 's![@]RUN[@]!../bus/test-bus.exe!' \
        < ../../dbus-1/test/tap-test.sh.in > test-bus.sh
FAIL: test-bus.sh 1 ../bus/test-bus.exe (exit status 1)
sed -e 's![@]RUN[@]!../dbus/test-dbus.exe!' \
        < ../../dbus-1/test/tap-test.sh.in > test-dbus.sh
FAIL: test-dbus.sh 1 ../dbus/test-dbus.exe (exit status 3)
PASS: test-shell.exe 1
PASS: test-shell.exe 2
PASS: test-shell.exe 3
PASS: test-shell.exe 4
PASS: test-shell.exe 5
PASS: test-shell.exe 6
PASS: test-shell.exe 7
PASS: test-shell.exe 8
PASS: test-shell.exe 9
PASS: test-shell.exe 10
PASS: test-shell.exe 11
PASS: test-printf.exe 1
PASS: test-printf.exe 2
PASS: test-printf.exe 3
PASS: test-printf.exe 4
PASS: test-printf.exe 5
PASS: test-printf.exe 6
PASS: test-printf.exe 7
ERROR: test-corrupt.exe - missing test plan
ERROR: test-corrupt.exe - exited with status 53
PASS: test-dbus-daemon.exe 1 /creds
PASS: test-dbus-daemon.exe 2 /processid
PASS: test-dbus-daemon.exe 3 /echo/session
PASS: test-dbus-daemon.exe 4 /echo/limited
PASS: test-dbus-daemon.exe 5 /no-reply/disconnect
PASS: test-dbus-daemon.exe 6 /no-reply/timeout
PASS: test-dbus-daemon.exe 7 /canonical-path/uae
PASS: test-dbus-daemon-eavesdrop.exe 1 /eavedrop/match_keyword/broadcast
PASS: test-dbus-daemon-eavesdrop.exe 2 /eavedrop/match_keyword/unicast_to_receiver
PASS: test-dbus-daemon-eavesdrop.exe 3 /eavedrop/match_keyword/unicast_to_sender
SKIP: test-fdpass.exe 1 /relay # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 2 /limit # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 3 /too-many/plus1 # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 4 /too-many/plus2 # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 5 /too-many/plus17 # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 6 /too-many/split # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 7 /flood/1 # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 8 /flood/half-limit # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 9 /flood/over-half-limit # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 10 /flood/limit # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 11 /odd-limit/minus1 # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 12 /odd-limit/at # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 13 /odd-limit/plus1 # SKIP fd-passing not supported on this platform
SKIP: test-fdpass.exe 14 /odd-limit/plus2 # SKIP fd-passing not supported on this platform
PASS: test-monitor.exe 1 /monitor/invalid
PASS: test-monitor.exe 2 /monitor/become
PASS: test-monitor.exe 3 /monitor/broadcast
PASS: test-monitor.exe 4 /monitor/forbidden-broadcast
PASS: test-monitor.exe 5 /monitor/unicast-signal
PASS: test-monitor.exe 6 /monitor/forbidden
PASS: test-monitor.exe 7 /monitor/method-call
PASS: test-monitor.exe 8 /monitor/forbidden-method
PASS: test-monitor.exe 9 /monitor/dbus-daemon
PASS: test-monitor.exe 10 /monitor/selective
PASS: test-monitor.exe 11 /monitor/wildcard
PASS: test-monitor.exe 12 /monitor/no-rule
PASS: test-monitor.exe 13 /monitor/eavesdrop
PASS: test-monitor.exe 14 /monitor/no-eavesdrop
PASS: test-loopback.exe 1 /connect/tcp
PASS: test-loopback.exe 2 /connect/nonce-tcp
PASS: test-loopback.exe 3 /message/tcp
PASS: test-loopback.exe 4 /message/nonce-tcp
PASS: test-loopback.exe 5 /message/bad-guid
PASS: test-marshal.exe 1 /demarshal/le
PASS: test-marshal.exe 2 /demarshal/be
PASS: test-marshal.exe 3 /demarshal/needed/le
PASS: test-marshal.exe 4 /demarshal/needed/be
PASS: test-refs.exe 1 /refs/connection
PASS: test-refs.exe 2 /refs/message
PASS: test-refs.exe 3 /refs/pending-call
PASS: test-refs.exe 4 /refs/server
PASS: test-relay.exe 1 /connect
PASS: test-relay.exe 2 /relay
PASS: test-relay.exe 3 /limit
PASS: test-syntax.exe 1 /syntax/path
PASS: test-syntax.exe 2 /syntax/interface
PASS: test-syntax.exe 3 /syntax/error
PASS: test-syntax.exe 4 /syntax/member
PASS: test-syntax.exe 5 /syntax/bus-name
PASS: test-syntax.exe 6 /syntax/signature
PASS: test-syntax.exe 7 /syntax/single-signature
PASS: test-syntax.exe 8 /syntax/utf8
PASS: test-syslog.exe 1 /syslog
SKIP: test-uid-permissions.exe 1 /uid-permissions/uae/root # SKIP cannot use alternative uid on Windows
SKIP: test-uid-permissions.exe 2 /uid-permissions/uae/messagebus # SKIP cannot use alternative uid on Windows
SKIP: test-uid-permissions.exe 3 /uid-permissions/uae/other # SKIP cannot use alternative uid on Windows
============================================================================
Testsuite summary for dbus 1.10.3
============================================================================
# TOTAL: 88
# PASS:  67
# SKIP:  17
# XFAIL: 0
# FAIL:  2
# XPASS: 0
# ERROR: 2

(In reply to Ralf Habacker from comment #22)
> sed -e 's![@]RUN[@]!../bus/test-bus.exe!' \
>         < ../../dbus-1/test/tap-test.sh.in > test-bus.sh
> FAIL: test-bus.sh 1 ../bus/test-bus.exe (exit status 1)
> sed -e 's![@]RUN[@]!../dbus/test-dbus.exe!' \
>         < ../../dbus-1/test/tap-test.sh.in > test-dbus.sh
> FAIL: test-dbus.sh 1 ../dbus/test-dbus.exe (exit status 3)

That's known, although I don't know why these are failing... analysis welcome. I think we should treat these as a separate bug, since they're very different.

> ERROR: test-corrupt.exe - missing test plan
> ERROR: test-corrupt.exe - exited with status 53

I'm not sure what's going on there - it didn't fail like that for me. What's in test-corrupt.log?

(That test works by making a D-Bus connection, and hacking the binary messages so they are syntactically invalid, and checking that libdbus at the other end does the right thing.)

> PASS: test-dbus-daemon.exe 1 /creds
> PASS: test-dbus-daemon.exe 2 /processid
(etc.)

This demonstrates that attachment 118990 [details] [review] is working. Without that patch, the Automake test driver would have just reported a "missing test plan" for all the tests.

Comment on attachment 118990 [details] [review]
[3/4] When running TAP tests, filter out trailing \r from Windows .exe

Review of attachment 118990 [details] [review]:
-----------------------------------------------------------------

looks good

Comment on attachment 118990 [details] [review]
[3/4] When running TAP tests, filter out trailing \r from Windows .exe

committed to dbus-1.10

(In reply to Simon McVittie from comment #23)
> (In reply to Ralf Habacker from comment #22)
> > sed -e 's![@]RUN[@]!../bus/test-bus.exe!' \
> >         < ../../dbus-1/test/tap-test.sh.in > test-bus.sh
> > FAIL: test-bus.sh 1 ../bus/test-bus.exe (exit status 1)
> > sed -e 's![@]RUN[@]!../dbus/test-dbus.exe!' \
> >         < ../../dbus-1/test/tap-test.sh.in > test-dbus.sh
> > FAIL: test-dbus.sh 1 ../dbus/test-dbus.exe (exit status 3)
> 
> That's known, although I don't know why these are failing... analysis
> welcome. I think we should treat these as a separate bug, since they're very
> different.
see Bug 92721

Comment on attachment 118983 [details] [review]
[4/4] refs test: reduce number of repeats under Wine

committed to dbus-1.10

Thanks. I think the only thing left to deal with here (ignoring Bug #92721 for the moment) is your test-corrupt failure?

(In reply to Simon McVittie from comment #28)
> Thanks. I think the only thing left to deal with here (ignoring Bug #92721
> for the moment) is your test-corrupt failure?

err:module:import_dll Library libgio-2.0-0.dll (which is needed by L"Z:\\home\\xxxx\\src\\dbus-1-autotools-cross-build\\test\\.libs\\test-corrupt.exe") not found
err:module:LdrInitializeThunk Main exe initialization for L"Z:\\home\\xxx\\src\\dbus-1-autotools-cross-build\\test\\.libs\\test-corrupt.exe" failed, status c0000135
ERROR: test-corrupt.exe - missing test plan
ERROR: test-corrupt.exe - exited with status 53

This was an installation issue and could be ignored.