automated tests update

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

automated tests update

Richie-4
Hello!

Wanted to update on the automation front. We are working on getting a regression environment set
up so that we can have all future tests run automated. We are 30% through with the task and plan 
to have almost 80% complete by the end of this week. 

The idea is to have these run in the night time so that by morning when we are in, the results are also
in and we can do the fixes wherever needed.

In due course of time, it will be nice if we have some volunteers who can take over this and run them
and do the necessary fixes so that there is no dependency on any specific time-zone-based teams.

Thanks,
Richie
_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: automated tests update

Gopal-2
Hi Richie,
 
I am aware of that currently we are using AdventNet QEngine for automating some of the test cases. But I am not clear, how can our external developer community use this tool in their local environment. Do you have any plans to get some more licenses for the sake of vtiger developer community?
 
Thanks,
Gopal
---
S.S.G.Gopal
skype: sripadag
ph: +1 877 788 4437
blog: http://gopal.vtiger.com




----[hidden email] wrote ----

Hello!

Wanted to update on the automation front. We are working on getting a regression environment set
up so that we can have all future tests run automated. We are 30% through with the task and plan 
to have almost 80% complete by the end of this week. 

The idea is to have these run in the night time so that by morning when we are in, the results are also
in and we can do the fixes wherever needed.

In due course of time, it will be nice if we have some volunteers who can take over this and run them
and do the necessary fixes so that there is no dependency on any specific time-zone-based teams.

Thanks,
Richie _______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: automated tests update

Jeff Kowalczyk
In reply to this post by Richie-4
Gopal wrote:
> I am aware of that currently we are using AdventNet QEngine for
> automating some of the test cases. But I am not clear, how can our
> external developer community use this tool in their local environment.
> Do you have any plans to get some more licenses for the sake of vtiger
> developer community?

I'm glad to see regression tests on the agenda, but we don't need non-free
tools to get this done.

Selenium, which I've mentioned before, runs recorded tests in a browser,
giving the most/only realistic test environment for javascript, etc.

Selenium tests can be recorded by a firefox plugin, and the test source
can be checked into subversion like any other source. This is what we
should use, IMHO.

http://openqa.org/selenium/

As far as running 'continuous integration testing'; some testing experts
have used buildbot and vmware/vnc integration to continuously and
automatically run unit and integration tests in a headless environment.

http://www.google.com/search?q=buildbot+selenium

http://agile.idyll.org/wiki/BuildBotTechnologyNarrative

http://agile.idyll.org/buildbot/

All this testing technology uses free software.

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: automated tests update

Matthew Brichacek
I haven't looked into Selenium yet but with JS does it work at the
source level only, does it use gecko to actually execute and evaluate
the JS or can it check for changes in the DOM like it does the source?
In 5.x there are quite a few places where the DOM is being manipulated
directly or indirectly and I don't think a source level verification
tool will catch most of those.

Maybe augmenting it with the scriptactulous unit test lib would work?
http://wiki.script.aculo.us/scriptaculous/show/UnitTesting


Matt

On Mon, 2006-07-17 at 15:01 -0400, Jeff Kowalczyk wrote:

> Gopal wrote:
> > I am aware of that currently we are using AdventNet QEngine for
> > automating some of the test cases. But I am not clear, how can our
> > external developer community use this tool in their local environment.
> > Do you have any plans to get some more licenses for the sake of vtiger
> > developer community?
>
> I'm glad to see regression tests on the agenda, but we don't need non-free
> tools to get this done.
>
> Selenium, which I've mentioned before, runs recorded tests in a browser,
> giving the most/only realistic test environment for javascript, etc.
>
> Selenium tests can be recorded by a firefox plugin, and the test source
> can be checked into subversion like any other source. This is what we
> should use, IMHO.
>
> http://openqa.org/selenium/
>
> As far as running 'continuous integration testing'; some testing experts
> have used buildbot and vmware/vnc integration to continuously and
> automatically run unit and integration tests in a headless environment.
>
> http://www.google.com/search?q=buildbot+selenium
>
> http://agile.idyll.org/wiki/BuildBotTechnologyNarrative
>
> http://agile.idyll.org/buildbot/
>
> All this testing technology uses free software.
>
> _______________________________________________
> Get started with creating presentations online - http://zohoshow.com?vt 

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: automated tests update

Jeff Kowalczyk
Matthew Brichacek wrote:
> I haven't looked into Selenium yet but with JS does it work at the
> source level only, does it use gecko to actually execute and evaluate
> the JS or can it check for changes in the DOM like it does the source?
> In 5.x there are quite a few places where the DOM is being manipulated
> directly or indirectly and I don't think a source level verification
> tool will catch most of those.

Selenium directly drives the browser as if the user were clicking it. The
demo http://www.openqa.org/selenium-core/demos.html illustrates the
concept.

> Maybe augmenting it with the scriptactulous unit test lib would work?
> http://wiki.script.aculo.us/scriptaculous/show/UnitTesting

Unit tests in the client and server-side source would be enhancements, to
be sure. However, I am skeptical about trying to retrofit unit test
coverage to a large code base. The community would have a hard time
achieving useful code coverage unless there were tests for *everything*,
starting from early in the project.

Selenium's end-result testing, and the fact that such tests can be
recorded for submission by even novice users, would be an efficient use of
limited community resources. If the testing fever strikes the community,
perhaps all new code could be required to have unit tests, and so on.

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: automated tests update

Matthew Brichacek
On Mon, 2006-07-17 at 18:53 -0400, Jeff Kowalczyk wrote:

> Matthew Brichacek wrote:
> > I haven't looked into Selenium yet but with JS does it work at the
> > source level only, does it use gecko to actually execute and evaluate
> > the JS or can it check for changes in the DOM like it does the source?
> > In 5.x there are quite a few places where the DOM is being manipulated
> > directly or indirectly and I don't think a source level verification
> > tool will catch most of those.
>
> Selenium directly drives the browser as if the user were clicking it. The
> demo http://www.openqa.org/selenium-core/demos.html illustrates the
> concept.
I tried this out and did a little digging in the Core and IDE.  From the
looks of it, Selenium can handle a _lot_ of the javascript testing we
would need, if not all of it.

>
> > Maybe augmenting it with the scriptactulous unit test lib would work?
> > http://wiki.script.aculo.us/scriptaculous/show/UnitTesting
>
> Unit tests in the client and server-side source would be enhancements, to
> be sure. However, I am skeptical about trying to retrofit unit test
> coverage to a large code base. The community would have a hard time
> achieving useful code coverage unless there were tests for *everything*,
> starting from early in the project.
>
> Selenium's end-result testing, and the fact that such tests can be
> recorded for submission by even novice users, would be an efficient use of
> limited community resources. If the testing fever strikes the community,
> perhaps all new code could be required to have unit tests, and so on.
I had thought of this before too but my experience in requiring unit
tests has not been good :).  In the CGL project we tried to enforce a
rule like this and it was shot down almost before it left the ground.
In a PoC environment like that it's understandable but I think even in
the case of vtiger the fever would have to be HOT to get everyone on
board.

Matt

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt