Bug 179256

Summary: Multiple http/wpt/beacon/contentextensions/ test are flaky.
Product: WebKit Reporter: Matt Lewis <jlewis3>
Component: New BugsAssignee: Chris Dumez <cdumez>
Status: RESOLVED FIXED    
Severity: Normal CC: achristensen, beidson, cdumez, commit-queue, ews-watchlist, ggaren, rniwa, ryanhaddad, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Archive of layout-test-results from ews106 for mac-sierra-wk2 none

Matt Lewis
Reported 2017-11-03 12:21:32 PDT
The tests: http/wpt/beacon/contentextensions/beacon-blocked.html http/wpt/beacon/contentextensions/beacon-redirect-blocked.html are flaky according to the flakiness dashboard on Sierra and High Sierra. They show signs of being flaky beyond the earliest run on the dashboard. Example build: https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r224405%20(5492)/results.html https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20(Tests)/builds/5492 History: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Fwpt%2Fbeacon%2Fcontentextensions%2Fbeacon-blocked.html https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Fwpt%2Fbeacon%2Fcontentextensions%2Fbeacon-redirect-blocked.html http/wpt/beacon/contentextensions/beacon-blocked.html diff --- /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/http/wpt/beacon/contentextensions/beacon-blocked-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/http/wpt/beacon/contentextensions/beacon-blocked-actual.txt @@ -1,5 +1,3 @@ -CONSOLE MESSAGE: line 9: Content blocker prevented frame displaying http://localhost:8800/WebKit/beacon/contentextensions/beacon-blocked.html from loading a resource from http://localhost:8800/WebKit/beacon/resources/beacon-preflight.py -CONSOLE MESSAGE: line 9: Beacon API cannot load http://localhost:8800/WebKit/beacon/resources/beacon-preflight.py. Resource blocked by content blocker -PASS Content extensions should be able to block beacons +FAIL Content extensions should be able to block beacons assert_false: sendBeacon should return false expected false got true http/wpt/beacon/contentextensions/beacon-redirect-blocked.html diff: --- /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/http/wpt/beacon/contentextensions/beacon-redirect-blocked-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/http/wpt/beacon/contentextensions/beacon-redirect-blocked-actual.txt @@ -1,4 +1,3 @@ -CONSOLE MESSAGE: Beacon API cannot load http://127.0.0.1:8800/WebKit/beacon/resources/beacon-preflight.py?allowCors=1&cmd=put&id=f470f43c-258c-4c82-b880-ace3bcdb211c&redirect_status=307&location=http%3A%2F%2F127.0.0.1%3A8800%2FWebKit%2Fbeacon%2Fresources%2Fbeacon-preflight.py%3FallowCors%3D1%26cmd%3Dput%26id%3Df470f43c-258c-4c82-b880-ace3bcdb211c&count=1. Blocked by content extension -PASS Content extensions should be able to block beacon redirects +FAIL Content extensions should be able to block beacon redirects assert_equals: Did not receive beacon expected 0 but got 1
Attachments
Patch (4.29 KB, patch)
2018-01-10 09:56 PST, Chris Dumez 		no flags
DetailsFormatted DiffDiff
Archive of layout-test-results from ews106 for mac-sierra-wk2 (2.69 MB, application/zip)
2018-01-10 11:50 PST, EWS Watchlist 		no flags
Details
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.
Ryan Haddad
Comment 1 2017-11-03 14:58:13 PDT
From stderr: 2017-11-03 13:37:34.083 WebKitTestRunner[89471:31874437] Rule list compilation failed: Unspecified error during compile.
Radar WebKit Bug Importer
Comment 2 2017-11-07 16:27:51 PST
<rdar://problem/35401978>
Ryan Haddad
Comment 3 2017-11-10 09:44:51 PST
Marked tests as flaky in https://trac.webkit.org/r224690
Chris Dumez
Comment 4 2018-01-09 15:36:00 PST
This is apparently not specific to Beacon. All content extensions tests are currently flaky and sometimes give the following error on stderr: "Rule list compilation failed: Unspecified error during compile.".
Chris Dumez
Comment 5 2018-01-09 15:43:06 PST
(In reply to Chris Dumez from comment #4) > This is apparently not specific to Beacon. All content extensions tests are > currently flaky and sometimes give the following error on stderr: > "Rule list compilation failed: Unspecified error during compile.". Alex added more logging recently and on recent builds we see: Content Rule List compiling failed: Moving file failed. 2018-01-09 12:15:40.338 WebKitTestRunner[91903:69125462] Rule list compilation failed: Unspecified error during compile.
Chris Dumez
Comment 6 2018-01-09 15:46:54 PST
(In reply to Chris Dumez from comment #5) > (In reply to Chris Dumez from comment #4) > > This is apparently not specific to Beacon. All content extensions tests are > > currently flaky and sometimes give the following error on stderr: > > "Rule list compilation failed: Unspecified error during compile.". > > Alex added more logging recently and on recent builds we see: > Content Rule List compiling failed: Moving file failed. > 2018-01-09 12:15:40.338 WebKitTestRunner[91903:69125462] Rule list > compilation failed: Unspecified error during compile. I also see: 2018-01-08 18:34:52.233 WebKitTestRunner[40704:65555369] Error in moveFile: Error Domain=NSCocoaErrorDomain Code=4 "“ContentRuleListSTFKD0” couldn’t be moved to “ContentExtensions” because either the former doesn't exist, or the folder containing the latter doesn't exist." UserInfo={NSSourceFilePathErrorKey=/var/folders/sz/cwvl9y7149bc1xwf4f_x1_k00000gn/T/WebKitTestRunners-pkI04D/ContentRuleListSTFKD0, NSUserStringVariant=( Move ), NSDestinationFilePath=/Users/buildbot/Library/WebKit/WebKitTestRunner/ContentExtensions/ContentExtension-TestContentExtensions, NSFilePath=/var/folders/sz/cwvl9y7149bc1xwf4f_x1_k00000gn/T/WebKitTestRunners-pkI04D/ContentRuleListSTFKD0, NSUnderlyingError=0x7fddf7074c80 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
Chris Dumez
Comment 7 2018-01-09 15:56:51 PST
(In reply to Chris Dumez from comment #6) > (In reply to Chris Dumez from comment #5) > > (In reply to Chris Dumez from comment #4) > > > This is apparently not specific to Beacon. All content extensions tests are > > > currently flaky and sometimes give the following error on stderr: > > > "Rule list compilation failed: Unspecified error during compile.". > > > > Alex added more logging recently and on recent builds we see: > > Content Rule List compiling failed: Moving file failed. > > 2018-01-09 12:15:40.338 WebKitTestRunner[91903:69125462] Rule list > > compilation failed: Unspecified error during compile. > > I also see: > 2018-01-08 18:34:52.233 WebKitTestRunner[40704:65555369] Error in moveFile: > Error Domain=NSCocoaErrorDomain Code=4 "“ContentRuleListSTFKD0” couldn’t be > moved to “ContentExtensions” because either the former doesn't exist, or the > folder containing the latter doesn't exist." > UserInfo={NSSourceFilePathErrorKey=/var/folders/sz/ > cwvl9y7149bc1xwf4f_x1_k00000gn/T/WebKitTestRunners-pkI04D/ > ContentRuleListSTFKD0, NSUserStringVariant=( > Move > ), > NSDestinationFilePath=/Users/buildbot/Library/WebKit/WebKitTestRunner/ > ContentExtensions/ContentExtension-TestContentExtensions, > NSFilePath=/var/folders/sz/cwvl9y7149bc1xwf4f_x1_k00000gn/T/ > WebKitTestRunners-pkI04D/ContentRuleListSTFKD0, > NSUnderlyingError=0x7fddf7074c80 {Error Domain=NSPOSIXErrorDomain Code=2 "No > such file or directory"}} TestController::platformConfigureViewForTest(const TestInvocation& test) uses [_WKUserContentExtensionStore defaultStore] which apparently ends up using the default store path, which ends up being shared by all WKTR instances that are running in parallel :(
Chris Dumez
Comment 8 2018-01-10 09:56:05 PST
Created attachment 330922 [details] Patch
EWS Watchlist
Comment 9 2018-01-10 11:50:25 PST
Comment on attachment 330922 [details] Patch Attachment 330922 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/6023313 New failing tests: http/tests/workers/service/postmessage-after-sw-process-crash.https.html
EWS Watchlist
Comment 10 2018-01-10 11:50:26 PST
Created attachment 330941 [details] Archive of layout-test-results from ews106 for mac-sierra-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews106 Port: mac-sierra-wk2 Platform: Mac OS X 10.12.6
Chris Dumez
Comment 11 2018-01-10 13:09:27 PST
Comment on attachment 330941 [details] Archive of layout-test-results from ews106 for mac-sierra-wk2 This failure cannot be related.
WebKit Commit Bot
Comment 12 2018-01-10 18:30:55 PST
Comment on attachment 330922 [details] Patch Clearing flags on attachment: 330922 Committed r226751: <https://trac.webkit.org/changeset/226751>
WebKit Commit Bot
Comment 13 2018-01-10 18:30:57 PST
All reviewed patches have been landed. Closing bug.
Note You need to log in before you can comment on or make changes to this bug.