View Single Post
Posts: 560 | Thanked: 422 times | Joined on Mar 2011
#59
This programme is such a good idea - standardise testing procedures. Is anyone maintaining it?

The UI kind of dies when it launches. If the software's busy, there's no indication of what it's up to. After a min or two it settles down.

Going to go through the questions...
Has proper bug tracker - can this be assessed automatically by running wget (or similar) over the package page?
Contains proper licence - where should the tester look for the licence? Package page, UI title bar... ? If the licence is in a standard place, again - could a script look for it?
No dubious content - Should this not be "No dubious content without clear indication" - surely naughty things can exist so long as the sensitve are aware? Or would such things infringe on nokia's willingness to host the item in the repos?
No broken functionality - OK, this is potentially massive and covers everything from doesn't work at all to small items not working in the way one thought they might. Perhaps breaking it down on a second page/tab a bit might help? This would give an opportunity to check for UI functionality and documentation.
Optified - is this testable automatically? Other than not wiping out one's rootfs, how does one actually "know" a package has been properly optified?
Resource usage - assuming the package doesn't max out cpu all the time, what's "reasonable". Some tasks are just intensive e.g. converting videos. Perhaps this could be automatically tested by checking what else can be done while the programme is being run?
Power use - again, so long as the programme doesn't short the device, what is reasonable?

For the login, it needs to be able to do so with bugs.maemo as well as maemo.org to write on the package page.

I don't know how/why but KISStester launches Opera for the web interactions. Probably these buttons just need redefining? Some buttons don't work.

I've suggested automating a few of the above tasks, which might seem a bit pointless because once one person has performed the test, surely it'll be the same for all users? Not on the N900! Everyone has a different device set up so the results of performance and storage tests would be different for every device. also, if everyone runs a standard test for licence and bugtracker location etc. , the developer(s) will be sure it's correct (or not!).

KISStester would work really well if it could be used with a Wiki page to flesh out the checklist. Also, am I missing something - is there a way to test -devel packages with it? On the same note, might be useful to be able to paste a comment from KISStester to the maemo.org downloads section for those packages that have made it to extras.

Maybe once tidied up, add to CSSU testing so more people provide useful comments and/or promote/fail packages?