More sensitive tests in DELVE
David J C MacKay
The current test in DELVE, for sum squared error loss,
has the unfortunate feature that if a method does really lousily
on a few test cases, the posterior standard deviation of
its mean loss is increased such that the test is weakened
and may well give a null result.
By the simple hack of modifying the loss function to
sum |\mbox{error}|^{p} with p<2, for example p=0.5
or p=0.1, it seems plausible that
the sensitivity of DELVE might be improved.
postscript (Cambridge UK).
postscript (Canada mirror).
David MacKay's:
home page,
publications.
bibtex file.
Canadian mirrors:
home page,
publications.
bibtex file.