|
»
»
Updating Tktest suite - GSoC project
|
View:
New views
11 Messages
—
Rating Filter:
Alert me
|
 |
Updating Tktest suite - GSoC project
by Ania Pawelczyk
::
Rate this Message:
Hello,
I'm Ania Pawelczyk, a student from Poland who participates in Google Summer of Code and works on updating Tk test suite. I'd like to ask for comments on my work, get to know whether I took right assumptions. I'd be also thankful for hints and suggestions on the other aspects that I omitted but I should concentrate on. I'd very welcome if people involved in upgrading test suite (like dgp, patthoyts, dkf, das...) would steer me on the right direction. As for now I took the following assumptions and concentrated on those aspects: 1. Failing tests I work by comparing results on different environments and my goal is to let tktests be passed on them. Currently I work on Slackware, Ubuntu and XP (on 2 machines and tomorrow I'll get a third one which, I think, is the minimum to do simultaneous testing). WinXP and Ubuntu pass rather well (Ubuntu has problems generally with tests where font constraint is checked whereas Win and Slack don't pass the font constraint at all). I'd install on the third one Vista and p.e. SuSE. I usually work by assuming that the test that passes on one machine but not on the others has probably "good" - I mean required - settings and the failing ones lack something or take wrong assumptions. I compare configurations and try to find what p.e. - wasn't set, but matters in the result and may have different default values - was somewhere wrongly assumed (like in http://www.assembla.com/flows/show/brq6hCsrar3BB4ab7jnrAJ) Is this alright? 2. Constraints.tcl I also wonder whether I could mess up with constraints.tcl to try them to be passed more likely (like fonts; i.e. http://www.assembla.com/flows/show/dM_zSCsdWr3zNyab7jnrAJ ). Or rather Should I only try to configure different desktop settings? Or maybe should I leave tests with skipped constraints as they are? 3. Individual test's structure Furthermore, should I also restructure tests? So that they'd follow the pattern that is presented in man tcltest page? P.e. message.test. I was advised to work on. As it passes all the tests on my current OSes and my friends ones, what should be done with it? This kind? # Current look test message-3.6 {MessageWidgetObjCmd procedure, "configure"} { message .m set result [list [catch {.m configure -foo} msg] $msg] destroy .m set result } {1 {unknown option "-foo"}} # I thought about test message-3.0 {MessageWidgetObjCmd procedure, "cget"} -setup { message .m } -body { .m cget } -cleanup { destroy .m } -returnCodes error -result {wrong # args: should be ".m cget option"} There're also are quite often widgets defined and set outside the test case and test case that uses the previous settings; Sometimes a few tests take advantage of that settings. Should I take care for this and turn it into (the same) individual test case setups? 4. Wiki page I added my project's wiki page where I put my ideas (marked(i)) and questions (marked(?)). http://www.assembla.com/flows/flow/a_KwcurkGr3ztnab7jnrAJ For the moment there're rfc for, briefly, my following problems: > Should I try to make the tktest constraints in a way that they'd be more likely to pass? {(?i) constraints.tcl - testConstraint fonts} > How to figure out in tcl if a definition was made during the compilation? {(?) frame.test frame-2.5} > Strange ubutnu's behaviour - wm geometry x and x coordination {(?) wm.test - wm-geometry-2.1 in Ubuntu } > RFC scrollbar.test scrollbar-3.42 I'd very welcome any comments for that or new threads with suggestions. I'm also available via email ania.pawelczyk@... Thanks in advance, Regards, Ania. ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
|
|
|
Re: Updating Tktest suite - GSoC project
by Donal K. Fellows-2
::
Rate this Message:
Ania Pawelczyk wrote:
> For the moment there're rfc for, briefly, my following problems: These are just my comments on some of the issues. I hope they're helpful to you... > > RFC scrollbar.test scrollbar-3.42 With lots of experience, a good test has the properties of being isolated (so that it can be run without other tests getting in the way) robust (so that it only tests what you want) and clear (so that when it fails, it tells you enough for you to figure out what failed). And it's *very* useful to be able to search on the test name; Tk's habit of doing 'test foo-5.[incr counter]' is something I'd like rid of even at the cost of making things more verbose. For the particular test that you were looking at there, it's probably better to split that into two tests (different numbers, different constraints) one which sets things up with testmetrics and the other of which doesn't. If I've understood the issue right, that is. :-) (The design of tests for the layout algorithms has got to be something of a black art, since you've got to figure out how to avoid brokenness from the environment and determine how to test what was intended instead of conducting experiments on what is actually done... Tricky!) > > How to figure out in tcl if a definition was made during the > compilation? {(?) frame.test frame-2.5} Firstly, that test *definitely* needs splitting so that it's two tests with different constraints. Curiously, that particular test needs to see something that's different between msys builds and VC++/BC builds. Arguably, that's a configuration bug. Something odd going on here, but I don't do a vast amount of Tk maintenance these days... Donal. ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
Re: Updating Tktest suite - GSoC project
by Ania Pawelczyk
::
Rate this Message:
Hi,
Thanks for all the replies. If you would have some time, I've got some tiny questions about the convention - they might have little matter, though I'd like to learn good style. Questions 3 & 4 refer to the 1.* Tktests that have a following pattern: (like in message.test) foreach test { {-aspect 3 3 bogus {expected integer but got "bogus"}} {-background #ff0000 #ff0000 non-existent [...] } { set name [lindex $test 0] test message-1.$i {configuration options} { ... 1. namespaces -should I create that? and names p.e. namespace eval testbutton. It's recommended on the manpage, but I haven't noticed any recently updated tcl or tk test using that. From manpage: A test file should be able to be combined with other test files and not interfere with them, even when configure -singleproc 1 causes all files to be evaluated in a common interpreter. A simple way to achieve this is to have your tests define all their commands and variables in a namespace that is deleted when the test file evaluation is complete. A good namespace to use is a child namespace test of the namespace of the module you are testing. 2. Which convention of package require tcltest 2.1 to choose? And tcltest 2.1 or 2? if {[lsearch [namespace children] ::tcltest] == -1} { package require tcltest 2.1 namespace import -force ::tcltest::* } or package require tcltest 2.1 3. For test 1.* {configuration option} which have all the same setups - which form to use? proc createButton {} { button .b -borderwidth 2 -highlightthickness 2 -font {Helvetica -12 bold} pack .b update } [...] } -setup { createButton .b configure -bd 4 } or proc createButton {option value} { button .b -borderwidth 2 -highlightthickness 2 -font {Helvetica -12 bold} -$option $value pack .b update } [...] } -setup { createButton bd 4} (or none of them:P) 4. For test 1.* {configuration option: ""} in the body I thought abut using .m cget -anchor instead of lindex [.m configure -anchor] 4 (sorry if I miss the concept :P) test message-1.3 {configuration option: "anchor"} -setup { createMessage .m configure -aspect 3 } -body { # lindex [.m configure -aspect] 3 ;# this was before .m cget -aspect } -cleanup { destroy .m } -result {3} Thanks in advance, Ania:) ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
Re: Updating Tktest suite - GSoC project
by Donal K. Fellows-2
::
Rate this Message:
Ania Pawelczyk wrote:
> 1. namespaces > -should I create that? and names p.e. namespace eval testbutton. It's > recommended on the manpage, but I haven't noticed any recently updated > tcl or tk test using that. I wouldn't bother. :-) Adding namespaces slightly improves the separation, but not by very much, and it makes much of the coding of the tests themselves less clear (as Tk prefers global variables). > 2. Which convention of package require tcltest 2.1 to choose? And > tcltest 2.1 or 2? > if {[lsearch [namespace children] ::tcltest] == -1} { > package require tcltest 2.1 > namespace import -force ::tcltest::* > } > or > package require tcltest 2.1 I'd just do: package require tcltest 2 ;# or 2.1 or whatever namespace import -force ::tcltest::* Trying to avoid the trivial amount of work involved in doing the above (which is what that [lsearch] stuff is about) is silly given that we're testing Tk, which does some very expensive operations indeed. :-) > 3. For test 1.* {configuration option} which have all the same setups - > which form to use? > proc createButton {} { > button .b -borderwidth 2 -highlightthickness 2 -font {Helvetica > -12 bold} > pack .b > update > } > [...] > } -setup { > createButton > .b configure -bd 4 > } > > or > proc createButton {option value} { > button .b -borderwidth 2 -highlightthickness 2 -font {Helvetica > -12 bold} -$option $value > pack .b > update > } > [...] > } -setup { createButton bd 4} > (or none of them:P) The first is better as it is clearer. I might even go for this: test .... -setup { button .b -borderwidth 2 ... pack .b update } -body { ... If you really want to use a helper procedure, do this: proc createButton args { button .b -borderwidth 2 ... {*}$args pack .b update } ... -setup { createButton -borderwidth 4 } ... You'll have to pick which you want to do yourself. > 4. For test 1.* {configuration option: ""} in the body > I thought abut using > .m cget -anchor > instead of > lindex [.m configure -anchor] 4 > (sorry if I miss the concept :P) Sounds fair enough to me, as the tests are of the options and not of the configuration machinery itself. That particular "idiom" dates from a long way back, before the cget method was added. FWIW, the cget method was added as part of Tk 4.0, so as you can see those tests have needed some work for a very long time! Donal. ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
Re: Updating Tktest suite - GSoC project
by Donald G Porter
::
Rate this Message:
Ania Pawelczyk wrote:
> I'm Ania Pawelczyk, a student from Poland who participates in Google > Summer of Code and works on updating Tk test suite. Thanks for taking on the project. I think you should establish for yourself and for the sake of others trying to offer help a clear sense of what you want to achieve. The Tk test suite is large, and the summer is short, so it's best if you focus efforts that most effectively lead toward your goals. From my perspective, the most important problems with the Tk test suite is that it fails its primary purpose - helping find and fix bugs in Tk. This is because 1) There are tests that fail, where that failure does not imply an actual bug in Tk; and 2) There are tests that fail, which do imply an actual bug in Tk, but don't clearly point to just where that bug is. I think the most valuable "updating" for the Tk test suite would try to tackle that. I would also aim for these corrections to get into releases of Tk starting with future patch releases of Tk 8.5. However, it's your project, so you choose the goals. Just communicate clearly to others what they are. > 1. Failing tests ... > I usually work by assuming that the test that passes on one machine but > not on the others has probably "good" - I mean required - settings and > the failing ones lack something or take wrong assumptions. I compare > configurations and try to find what p.e. > - wasn't set, but matters in the result and may have different default > values > - was somewhere wrongly assumed (like in > http://www.assembla.com/flows/show/brq6hCsrar3BB4ab7jnrAJ) > Is this alright? I think it's fair to assume every test probably passed for the developer who first added it in their usual development environment. If a test is failing now, that can be for many reasons. A common reason is that there was an unstated assumption made by the test which is no longer true, or not true on a different platform, etc. Test constraints can deal with that situation, assuming the test is sensible in the first place. That's really the most difficult part of test suite maintenance, especially many years after the fact: trying to discern just what was being tested, and whether it was a sensible test in the first place. As we often observe in the chatroom, the test suites include both tests and experiments. Only the tests are really useful. There's little point in tidying up the experiments. Much effort for little value. > 3. Individual test's structure > Furthermore, should I also restructure tests? So that they'd follow the > pattern that is presented in man tcltest page? Here is where clarity of your goals is important. I would love to see this task done. Getting the test suite(s) updated to use the features of tcltest 2 would be great! (Including tcllib even better!). But in the end, this is a semi-mechanical conversion process that doesn't make the test suites fundamentally better than they are now. So, I say do this if you decide this is your goal. Or if it's not your goal, do it if you can do it with reasonably small effort so that progress toward your goals isn't impeded. Otherwise, don't worry about it, and just be sure to use modern conventions in any new tests you create. > 4. Wiki page > I added my project's wiki page where I put my ideas (marked(i)) and > questions (marked(?)). I think it's a mistake to set this work aside somewhere. If you're going to work on updating the Tk test suite, let's get you registered as a Tk maintainer, and you can start committing to the HEAD and core-8-5-branches. You'll get a lot more and better review of your work if you're hacking the actual Tk sources than you will over in your own space somewhere. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@... Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
Re: Updating Tktest suite - GSoC project
by Donald G Porter
::
Rate this Message:
Ania Pawelczyk wrote:
> 1. namespaces > -should I create that? and names p.e. namespace eval testbutton. It's > recommended on the manpage, but I haven't noticed any recently updated > tcl or tk test using that. Examples of Tcl *.test files using the recommended technique: chanio, cmdMZ, encoding, error, fileSystem, iogt, io, main, parse, source. This is more important in the Tcl test suite, where the default mode is -singleproc 0. The namespace separation helps avoid bad surprises on occasional tests of the -singleproc 1 mode. The Tk test suite defaults to -singleproc 1 (no one wants to pay Tk process startup costs for every *.test file) so any troublesome conflicts are going to get detected quickly. In that circumstance, namespace protection is less important. > 2. Which convention of package require tcltest 2.1 to choose? And > tcltest 2.1 or 2? > if {[lsearch [namespace children] ::tcltest] == -1} { > package require tcltest 2.1 > namespace import -force ::tcltest::* > } > or > package require tcltest 2.1 Use this: package require tcltest 2.2 namespace import ::tcltest::* I'm assuming your work is aimed at the test suite to go with Tk 8.5.* and later. This implies a Tcl version of at least 8.5.0 as well. Version 2.2 of tcltest has been available since Tcl release 8.4.0. The misfeature of [namespace import] that lead to all the -force-ing has been fixed since Tcl 8.4.12. Code for the future, not the past. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@... Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
Re: Updating Tktest suite - GSoC project
by Ania Pawelczyk
::
Rate this Message:
I think you should establish for yourself and for the sake of others I think the same! I thought about taking this as my task: _______________________________________________________________ 1. Firstly bringing all tests to tcltest 2.0 syntax >to be sure that the failure is not caused by dependences. >to provide legibility, cohesion and get rid of a great mess with the tcltest package. (Rolf Ade) >to learn the tests, which would be helpful in the next project step that would be: 2. Eliminating situations when the test fails because there were made wrong assumptions.(I don't know how much of this I'll manage to do). _______________________________________________________________The design of tests for the layout algorithms has got to be something of a The real problems with Tk's test suite are that it is highly which is (I suppose) what DGP yesterday said:
As for>From my perspective, the most important problems with the Tk test Q: ...trying to deal with the above actually bases on finding failing test and then (if there is no wrong assumption in widget's configuration and if the failure isn't caused by global dependence) I can find the problem solution it in the sources? (I assume that the test cases themselves are alright and my job is to > eliminate wrong assumptions (p.e. by setting omitted configurations, by using constraints or (as you let me know) eliminating global dependences) > or simply inform what was reason, if something not connected with test case, caused the failure.) 2) There are tests that fail, which do imply an actual bug in Tk, but don'tQ: I'm not sure what I should do in this case... Is it possible to give an example of this situation and it's solution? I've directed my first steps of bringing all tests to tcltest 2.0 toward message.test, as I was advised to, and button.test. I'd really appreciate any comment on that, as this are my toddler steps. >4. Wiki page I think it's a mistake to set this work aside somewhere. If you're I'm only a novice in Tcl/Tk and giving me the right to commit on SF could be pretty huge for me:P Maybe overwhelmingly huge; I think I should first get into Tcl/Tk, gain some experience, survive a flux of criticism, grind my style... and then, possibly, I'll be prepared :P Some time ago I created svn for the project. I thought about sending there my code until it's not verified/accepted. This is where I've put the current code (message.test and button.test) http://svn.assembla.com/svn/TkTests/ (button.test is not complete, but I'd like to know if it tends toward good direction) (it took me unbelieveably long to create this mail and this is why I have done so little progress :P). I thought that maybe this thread would be a proper place to comment... so to give the outline: GENERALLY: (main changes, assumptions; questions (Q)) (I'd like to describe even tiny points to be sure I don't miss the idea.) 1. Global dependences (- so the main goal to destroy) I removed all global assumptions. Every widget created - is created and destroyed in the a case. In button.test there was also need to unset the buttons' variables. There're are no global variables, dependences and interferences. I resigned from helper procedures for setup. 1.1 -setup I don't have certainty which pattern is the most suitable? (During implementing I was often making up my mind so setups are not yet consistent - there are in one of three folloing patterns - I'll fix it after choosing the right form.) I was suggested by tcltest man page where test example-1.1 {test file existence} -setup { set file [makeFile {} test] } -body { file exists $file } -cleanup { removeFile test } -result 1 but tcltests (from tcl test suite) that're written in tcltest 2 prefer the third idea. Idea1: # putting into body everything but result; I suppose it's a bad idea:P If there's an error we know too little test message-1.1 {configuration option: "anchor"} -setup { message .m -borderwidth 2 -highlightthickness 2 -font {Helvetica -12 bold} pack .m update .m configure -anchor w } -body { .m cget -anchor } Idea2: # the features important for the result are put into body. But sometimes it could be # needed to know p.e. what widget was created test message-1.1 {configuration option: "anchor"} -setup { message .m -borderwidth 2 -highlightthickness 2 -font {Helvetica -12 bold} pack .m update } -body { .m configure -anchor w .m cget -anchor } Idea3 test message-1.1 {configuration option: "anchor"} -body { message .m -borderwidth 2 -highlightthickness 2 -font {Helvetica -12 bold} pack .m update .m configure -anchor w .m cget -anchor } Q: Which pattern is the most suitable? Q: Do pack widgets? In the original tests they're sometimes packed. Keep that the same, pack always or resign from displaying? 1.2 Configuring widget In the originals there were added options to database. In fact, they had no influence on any test result. They were, as described in button.test # Create entries in the option database to be sure that geometry options # like border width have predictable values. option add *Button.borderWidth 2 [...] Q: Keep those options set while creating a widget? (they don't have any influence) 1.3 cleanup - destroy I destroy widgets even if they have no right to be created. test button-2.7 {ButtonCreate - bad window name} -body { button foo } -cleanup { destroy foo } -returnCodes {error} -result {bad window path name "foo"} Q: Do so? 1.4 cleanup - unset I've noticed that in cleanup variables should be unset, as p.e. when I run button-3.55 and unsetting in button-3.54 wasn't done - it was passed when I run as a single test case - it failed when I run all tests, Q: Is that thinking correct? or maybe do unsetting and destroying, (with catching errors) in the setup? 2. Test cases 2.1 I split the test cases that both catch an error and obtain informations (like winfo, errorInfo) to detect an error by -returnCodes. Previous: test message-2.3 {Tk_MessageObjCmd procedure} { list [catch {message .s -gorp dumb} msg] $msg [winfo child .] } {1 {unknown option "-gorp"} {}} Idea: test message-2.4 {Tk_MessageObjCmd procedure} -body { message .s -gorp dump } -returnCodes {error} -result {unknown option "-gorp"} test message-2.5 {Tk_MessageObjCmd procedure} -body { catch {message .s -gorp dump} winfo child . } -result {} Q: Appropriate? 2.2 Checking if no error There're some test cases that only expect the return code that should be {ok}. However, I need to have a -result to compare with. I've thought about two options (plus one rather pointless). IMHO as the secnd one's construction is analogical to the ones that return an error code, I'd use that one. Hovever, I don't know if -match {glob} -result {*} ;# means result does'n matter is any sensible. The third Idea would be simply setting an option like in tests 1.* Previous: test button-4.7 {ButtonWidgetCmd procedure, "cget" option} { catch {.l cget -disabledforeground} } {0} Idea1: test button-3.7 {ButtonWidgetCmd procedure, "cget" option} -setup { label .l } -body { catch {.l cget -disabledforeground} } -cleanup { destroy .l } -result {0} Idea2: test button-3.7 {ButtonWidgetCmd procedure, "cget" option} -setup { label .l } -body { .l cget -disabledforeground } -cleanup { destroy .l } -returnCodes {ok} -match {glob} -result {*} ;# means result does'n matter Idea3: The last idea is to set an otion value and expect it in the result, but then it would be nothing but 1.* {configuration option: ""} test case. Q: What to choose? 2.3 Tests 1.* {configuration option: ""} To obtain the result I use {cget -option} instead of {lindex [.m configure -anchor] 4} "as the tests are of the options and not of the configuration machinery itself." (DKF) Though in the later test case there is cget option checking: as this is done before, but there's no lindex [.m configure -option] 4 checking, this time I used configure instead of cget. This is the test case I'm describing: test button-4.6 {ButtonWidgetCmd procedure, "cget" option} { .b configure -highlightthickness 3 .b cget -highlightthickness } {3} 2.4 test button-1.* {configuration option) - constraint constraints testImageType is not needed in most test-1.* cases 3. Descrptions Q: As there're the same test cases in diffrent tests, I thought about unifying the way that cases are described. Only oes that matter? (I haven't done that yet). Ania:) P.S. Examples of Tcl *.test files using the recommended technique: chanio,I must have been blind:P sorry. P.S.2 As we often observe in If "little value", then just out of curiosity: Q: what would you call an experiment? What is the difference? Could you give me an example? P.S.3 I'm simply curious about: Q: Do the test cases test everything (or most) that is needed? Is it the widget/procedure's creator/mantainer job to create test case and determine the result? ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
Re: Updating Tktest suite - GSoC project
by Donald G Porter
::
Rate this Message:
Ania Pawelczyk wrote:
> 1. Firstly bringing all tests to tcltest 2.0 syntax If that's the first goal you've set for yourself, that's fine, and I'm interested to review your progress and guide a bit. > to be sure that the failure is not caused by dependences. > to provide legibility, cohesion and get rid of > a great mess with the tcltest package. (Rolf Ade) I don't see a clear connection between making more use of tcltest 2 capabilities and these subpoints, FWIW >>> I added my project's wiki page ... >> I think it's a mistake to set this work aside somewhere. If you're >> going to work on updating the Tk test suite, let's get you registered >> as a Tk maintainer, ... > I'm only a novice in Tcl/Tk and giving me the right to commit on SF > could be pretty huge for me:P Maybe overwhelmingly huge; I think I > should first get into Tcl/Tk, gain some experience, survive a flux of > criticism, grind my style... and then, possibly, I'll be prepared :P If you're unwilling to take the full maintainer plunge right away, ok, but at least submit your work as patches to the Tktoolkit Tracker. That where contributions to Tk get reviewed in the usual way. Doing your work over on some private place is just a recipe for not getting as many reviewers. Reviewing actual proposed revisions to actual files will also bring up issues in a more natural way than trying to answer a lot of "big principle" questions up front, I think. So I'm going to snip a lot of that and let matters come up in the patches you offer. That will also take a lot of this out of the TCLCORE mailing list where it burdens all readers, and into the followup traffic of particular Tracker entries, where the audience is more focused. >> As we often observe in >> the chatroom, the test suites include both tests and experiments. > Q: what would you call an experiment? What is the difference? A test should exist for a purpose. Many tests get written (or should get written) to verify that some segment of C code produces the correct result. A test is a script designed to exercise a particular bit of program logic, and which is written so that the result indicates whether that bit of program logic operated correctly. This usually means that the expected result can be written down before ever running the script. When running the test produces the correct expected result, you have confidence you've coded that bit of logic correctly. An experiment is when a script is written, perhaps for a targeted testing purpose, perhaps not. Then the script is run, and the result is recorded and used as the expected result for the test. From then on the test is just a check that the result has not changed from the first empircal result. That's not a useless check; detecting changed results has some value. But it's not the same as a test which is aimed at more logical correctness verification. > Q: Do the test cases test everything (or most) that is needed? Is it the > widget/procedure's creator/mantainer job to create test case and > determine the result? The theory is that the same maintainers who maintain a file like tkFoo.c will also maintain the collection of tests, foo.test, for the code in that file. And that theory is mostly true as far as it goes, but it only goes so far when we have a small number of Tk maintainers and each of them has only a small amount of effort to devote to Tk. -- | Don Porter Mathematical and Computational Sciences Division | | donald.porter@... Information Technology Laboratory | | http://math.nist.gov/~DPorter/ NIST | |______________________________________________________________________| ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
Re: Updating Tktest suite - GSoC project
by Jeff Hobbs
::
Rate this Message:
Ania Pawelczyk wrote:
> I think you should establish for yourself and for the sake of others > trying to offer help a clear sense of what you want to achieve. (DGP) > > > I think the same! I thought about taking this as my task: > _______________________________________________________________ > 1. Firstly bringing all tests to tcltest 2.0 syntax > >to be sure that the failure is not caused by dependences. > >to provide legibility, cohesion and get rid of > > a great mess with the tcltest package. (Rolf Ade) > > >to learn the tests, which would be helpful in the next project step > that would be: > 2. Eliminating situations when the test fails because there were made > wrong assumptions. > (I don't know how much of this I'll manage to do). > _______________________________________________________________ I said this to Ania in a separate email, but I'll reiterate here because it represents the value to not only the Tcl/Tk core community, but to the student working on it (in this case Ania). While this may seem minimal with just 2 points, you would break down the reasoning and value into multiple points. It's not just having the new test structure format, it is taking advantage of what that provides. Improved clarity, separation of tests, robustness to the test suite, etc. On the 2nd, you learn a lot about the language, test automation principles, platform discrepancies and more. GSOC is just a summer commitment. Consider that the tests suite for Tk is over 10 years of amalgamated stuff. Having the opportunity for someone to give it an update with consistent style and purpose is very valuable. > The real problems with Tk's test suite are that it is highly > dependent on the platform that you test on (i.e. it's clean on only one > or two machines in the whole world!) and that it often doesn't check for > what *should* be the answer, but rather just what *is* the answer. > Fixing those things (or at least making a good step on them) will be a > good task. (DKF, some time ago, during application time) This is sadly true. I don't expect you to fully succeed in the time alloted to fully harden the test suites for the variety of platforms it could be run on. However, moving consistently towards that path should provide a framework against which to improve the longevity and reduce the brittleness overall. > Q: ...trying to deal with the above actually bases on finding failing > test and then (if there is no wrong assumption in widget's configuration > and if the failure isn't caused by global dependence) I can find the > problem solution it in the sources? If it isn't, don't hesitate to ask. > As for > > 2) There are tests that fail, which do imply an actual bug in Tk, > but don't clearly point to just where that bug is. > > Q: I'm not sure what I should do in this case... Is it possible to give > an example of this situation and it's solution? Find a 'knownBug' case. Some may be registered in our bug db, some may predate the SF bug db, some may already be fixed and not verified, ... > Q: Which pattern is the most suitable? > Q: Do pack widgets? In the original tests they're sometimes packed. > Keep that the same, pack always or resign from displaying? Sometimes pack is required to force the widget to consider geometry calculations that determine size. > 1.2 Configuring widget > In the originals there were added options to database. In fact, they had > no influence on any test result. They were, as described in button.test > # Create entries in the option database to be sure that geometry options > # like border width have predictable values. > option add *Button.borderWidth 2 [...] > Q: Keep those options set while creating a widget? (they don't have any > influence) This is meant to enforce consistency on platforms that may have alternate default parameters. For example, this way we know borderwidth will always be 2 for geom calculations. Otherwise you are having to give variant answers depending on the platform, all because of a variant borderwidth default that really isn't pertinent to the test case. > 1.3 cleanup - destroy > I destroy widgets even if they have no right to be created. > test button-2.7 {ButtonCreate - bad window name} -body { > button foo > } -cleanup { > destroy foo > } -returnCodes {error} -result {bad window path name "foo"} > Q: Do so? A taste choice IMO, as long as you are clean for the next test. > 1.4 cleanup - unset > I've noticed that in cleanup variables should be unset, as > p.e. when I run button-3.55 and unsetting in button-3.54 wasn't done > - it was passed when I run as a single test case > - it failed when I run all tests, > Q: Is that thinking correct? or maybe do unsetting and destroying, (with > catching errors) in the setup? This has blown test cases before (especially an array 'foo' that was later referenced as a scalar). Clean is good, but you often balance effort against value. > 2. Test cases > 2.1 I split the test cases that both catch an error and obtain > informations (like winfo, errorInfo) to detect an error by -returnCodes. > Previous: > test message-2.3 {Tk_MessageObjCmd procedure} { > list [catch {message .s -gorp dumb} msg] $msg [winfo child .] > } {1 {unknown option "-gorp"} {}} > Idea: > test message-2.4 {Tk_MessageObjCmd procedure} -body { > message .s -gorp dump > } -returnCodes {error} -result {unknown option "-gorp"} > test message-2.5 {Tk_MessageObjCmd procedure} -body { > catch {message .s -gorp dump} > winfo child . > } -result {} > Q: Appropriate? Yes, this is indeed a key feature in tcltest 2.0, that you can separate return info from error code. > 2.2 Checking if no error > Q: What to choose? It's OK to have no result. Some things test crash situations - we just hope not to crash. ;) > 3. Descrptions > Q: As there're the same test cases in diffrent tests, I thought about > unifying the way that cases are described. Only oes that matter? (I > haven't done that yet). Not a major item. I believe all the rest were answered. Regards, Jeff ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
|
|
Re: Updating Tktest suite - GSoC project
by Ania Pawelczyk
::
Rate this Message:
Hi,
I'm commiting my work to the HEAD on SF, as I was advised to do. I'd be thankful for some review and advices. By now I've managed to commit the following files: bell.test bgerror.test bind.test bitmap.test border.test button.test cursor.test entry.test message.test spinbox.test I'd also be grateful for any comment on my future commits; and thanks Pat Thoyts for pointing my mistake. Thanks in advance, Ania ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Tcl-Core mailing list Tcl-Core@... https://lists.sourceforge.net/lists/listinfo/tcl-core |
| Free Forum Powered by Nabble | Forum Help |