Ideas for vtiger architecture moving forward

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Ideas for vtiger architecture moving forward

Matthew Brichacek
I have some ideas for cleaning up the structure of vtiger as we move
forward.  I am just going to lay them out here, let me know what you
think.

My basic idea comes down to writing wrapper classes that are used to
abstract most of the unnecessary functions from developers and instead
provides a uniform API to access modules, permissions, input, etc.

The idea comes in three pieces.  Common UI classes, most of which is
already done with the new UI.  A "mainframe" engine similar to the
joomla $mainframe.  And last but certainly not least is an entity engine
that would abstract most of the entity details from developers and truly
provide the unified API.

Lets start with the big one, Entity Engine:
The entity engine would be accomplished in three steps.  The first step
is to move all common variables and functions into the EntityEngine
class (which extends CRMEntity).  The entity engine would go into the DB
to populate these with the correct info.  For example tab_name,
tab_name_index and all the other most common variables would be stored
in the db and populated when the entity engine has a request for that
module.  The next step would be to create common set of functions in
this class that would be exposed to the developer.  The initial
functions I can think of are things like get/set functions for different
field values, etc.  The last step to the entity engine would be an XML
install system.  If a developer came along and wanted to create a simple
data collection module, like the current campaigns module, they would
simply create an XML file that defines all the fields to be displayed,
what relationship types are allowed, etc.  The entity engine would then
create the correct tables, populate the needed tables for the entity
engine (the ones holding tab_name, etc) and register the new entity type
with the system.

This leads into the UI system because if the developer is just creating
the XML file, how are common views displayed (list, detail, edit,
relatedlists)?

The UI system would be a set of classes to extend smarty and again
abstract the gory details of the internal workings from the developer.
VTigerUI::DetailView($entity) would follow a flow:
1) Display all common elements (header, etc):
$this->createTigerHeader();
2) Display the common 2 tab display for edit/detail/related views
$this->getCommonUI("DetailView");
3) Create a block for each block type in the DB, and create the fields
for it as well.
$this->assign("NUM_BLOCKS",sizeof($entity->blocks));
foreach($entity->blocks as $blockid=>$fields) {
        $block_fields = array();
        foreach($fields as $fieldname=>$field_data) {
                $block_fields = $this->CreateEditField($fieldname,$field_values); //
creates new edit fields based on uitype and returns an array as
$array[sequence_number] = "HTML output"
        }
        $this->assign("BLOCK_".$blockid,$this->addBlock($blockid,
$block_fields));
}
4) Display data :)

And just like magic, the ListView, DetailView, EditView and
CallRelatedList files can all go away and these common views can be
managed in a much easier way.

Next, the mainframe.  In joomla (as far as I understand) the mainframe
is meant to handle all GET, POST, REQUEST and SESSSION variables and it
also sanitizes that input before giving it to the developer
($mainframe->get_input("REQUEST","data")).  On top of this it handles
some user security and other things.  In vtiger I could see it being
used for many similar things.  Take for example the CreateDetailField()
function call above.  Inside of this call would be:
if($mainframe->is_allowed("Detail",$field->id)) {
        stuff..
else
        return;

There are still other details to be worked out, but you see that this
isn't a _really_ large change for the flexibility it will give you in
the system.

After it's all said and done you should have two sets of documents that
a developer would have access to, the module developers document that
would be a document outlining the three pieces discussed above and then
the core developer document that would document all the core back-end
functions (everything in utils/).

Of course there would be a way to overwrite these methods so that
developers can still create non-entity based modules (webmails,
calendar, rss, etc), but it should make life much easier for module
developers who follow the entity structure in vtiger.

What do you think?  Am I nuts?  Does it take too much away from the
developer in the name of standardization?  To me it feels like the next
natural step to cleaning up the system and creating defined layers as
far as security, display and entities are concerned.  It should cut down
on all the unnecessary code re-creation in the system as well.


Matt

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Dennis Grant

> My basic idea comes down to writing wrapper classes that are used to
> abstract most of the unnecessary functions from developers and instead
> provides a uniform API to access modules, permissions, input, etc.

Oooh.

> The idea comes in three pieces.  Common UI classes, most of which is
> already done with the new UI.  A "mainframe" engine similar to the
> joomla $mainframe.  And last but certainly not least is an entity
engine
> that would abstract most of the entity details from developers and
truly
> provide the unified API.

To be honest, I think this is moving in the wrong direction.

If everybody used the same vanilla VTiger installation, this would make
a lot of sense - one common UI for the entire set of users, and a much
cleaner codebase for developers. Happy happy for everybody.

Unfortunately, a lot of us are NOT using vanilla VTiger; we have to
customize the application to meet the needs of our user base. And a lot
of the time, our users really really want changes that may seem utterly
trivial from a developer standpoint, but for whatever reason, they
REALLY want it that way.

And the customer is always right, so we have to do it.

I don't, for example, want to see something like where Accounts and
Contacts share the same DetailView module - because in my setup, there
are large differences between the DetailView we do for Accounts, and the
DetailView we do for Contacts.

It is a whole lot easier to deal with each module having its own code,
even if that code is 95% common with other modules in a vanilla install,
because that way, if I'm working on the Contacts DetailView, I don't
have to worry about changes spilling over into the operation of another
module.

Yes, it makes changing common code more difficult, 'cause now I have to
drill down into each submodule and make the same change over and over
again, and I can certainly see where modularizing and sharing code makes
that job a lot easier. But the assumption it is based on, while true of
a vanilla install, is NOT true in a customized install environment.

I'd prefer to see every module live in its own subdirectory of the
modules directory, much as is does in 4.x, but more rigorously (ie,
separate Sales Orders from Purchase Orders, don't group them in Orders)
and each module should have its own, atomic codebase.
 
We're doing things in the field that are far more advanced than just
custom fields (although we make use of those too) and we need the
flexibility that comes with exposing the dirty details.

Same thing with database queries. It's tempting I know to try and
abstract all those away with wrapper functions (and there are some
wrapper functions that are OK - something like
getContactEmail($contactID) or getContactAccount($contactID) or
isDeleted($crmID) are all simple and handy and useful for simplifying
out some of the database access overhead.

But as soon as you start getting into more complex queries, I want to
see the SQL right there in the code. I need to understand EXACTLY what
is going on, and I may need to muck with the query; especially if I have
changed/extended the database/table.

For example, our users had a need for Quotes to preserve the order in
which Products were added to the Quote. The intent is that they wanted
the PDF export of a Quote to do something like this:

Big expensive item 1
Option A for item 1
Option B for item 1
Option C for item 1
Big expensive item 2
Option A for item 2
Option C for item 2

But as the items came out of the database in the order in which they
were added to the database, what they got once they saved it might be
something like:

Option C for Item 2
Option A for item 1
Big Expensive item 2
...etc

>From a developer point of view, that's OK, 'cause all the items are on
there and the total is right, so what's the big deal? Well to them, this
WAS a big deal - a really big deal. They wanted this changed, and my
boss wanted it to happen NOW; so I extended the QuoteProductsRel to
include a sequenceNumber.

Then they wanted to be able to add headings, like this:

Heading 1
Big expensive item 1
Option A for item 1
Option B for item 1
Option C for item 1

Heading 2
Big expensive item 2
Option A for item 2
Option C for item 2

So I extended QuoteProductsRel to include a heading field that, if
populated, would print on the line preceding the product it was
associated with.

Etc etc etc.

This sort of stuff is WAY easier if the code to do it is right there,
and not buried in a common function somewhere (like a lot of the UI code
is, where you have the Mother of all If Statements lurking in utils.php
that has to account for *every single module* before it presents the UI.

I'd like to see some standardization on the module file structure
though:

modules/Foo/EditView.php for the edit UI for the module object
modules/Foo/DetailView.php for the detailed view of the object
modules/Foo/Save.php for saving the object to the DB
modules/Foo/CreatePDF.php for creating a PDF version of the object
modules/Foo/UIElements.php for the common UI functions (that are now
stuffed in include/utils.php
modules/Foo/BarPopup.php if there is a popup window to select associated
Bar objects

etc.

It makes dealing with modules easier if we know where the various
functions are kept.

DG

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Allan Bush
Proper use of a OO design would solves this for both of you.

All the common code and default module behavior should be moved into
the a set of default classes or "engines" as Matthew is calling them.
Then for each module we inherit the classes and make a couple method
calls (ideally this is abstracted as well so to create a module all
you do is create a class which inherits the base class) which
generates the required database actions / html page.  If you want
something different then the default, like Dennis describes, you
simply extent the required method in your inherited class.  This way
the common code is abstracted and only needs to exist once while the
module specific code is in the module for easy modification.

The code is already partly structured this way (it's just not used
properly).  There's already a base class CRMentity which every module
inherits from.  What needs to be done is to clean up the CRMentity -
module relationship so that only methods common to all modules and
their default behavior is defined in CRMentity and only things which
need to be changed are defined in the module class.  Then we need to
create a class for each page type (again some of this already exists.
ala inculdes/Listview.php, but here it isn't used at all) with the
default behaviors of each page type.  Now we have an option, either we
have each module inherit these page type classes (making changes as
needed ala CRMentity) and have a common set of calls to render each
page type or we create a SMALL file in each module with a couple of
calls to the page type class to render the page and we substitute new
functions as needed for customizations.  Personally I think the first
option here is the better design, but unless you get it right it could
become difficult to modify the parts of the page you want without
re-witting a lot of the base class (should be avoided at all cost or
we're right back in the mess we are in now).

That was a bit of a long winded explanation, let me know if anyone
requires clarification.

On 7/14/06, Dennis Grant <[hidden email]> wrote:

>
> > My basic idea comes down to writing wrapper classes that are used to
> > abstract most of the unnecessary functions from developers and instead
> > provides a uniform API to access modules, permissions, input, etc.
>
> Oooh.
>
> > The idea comes in three pieces.  Common UI classes, most of which is
> > already done with the new UI.  A "mainframe" engine similar to the
> > joomla $mainframe.  And last but certainly not least is an entity
> engine
> > that would abstract most of the entity details from developers and
> truly
> > provide the unified API.
>
> To be honest, I think this is moving in the wrong direction.
>
> If everybody used the same vanilla VTiger installation, this would make
> a lot of sense - one common UI for the entire set of users, and a much
> cleaner codebase for developers. Happy happy for everybody.
>
> Unfortunately, a lot of us are NOT using vanilla VTiger; we have to
> customize the application to meet the needs of our user base. And a lot
> of the time, our users really really want changes that may seem utterly
> trivial from a developer standpoint, but for whatever reason, they
> REALLY want it that way.
>
> And the customer is always right, so we have to do it.
>
> I don't, for example, want to see something like where Accounts and
> Contacts share the same DetailView module - because in my setup, there
> are large differences between the DetailView we do for Accounts, and the
> DetailView we do for Contacts.
>
> It is a whole lot easier to deal with each module having its own code,
> even if that code is 95% common with other modules in a vanilla install,
> because that way, if I'm working on the Contacts DetailView, I don't
> have to worry about changes spilling over into the operation of another
> module.
>
> Yes, it makes changing common code more difficult, 'cause now I have to
> drill down into each submodule and make the same change over and over
> again, and I can certainly see where modularizing and sharing code makes
> that job a lot easier. But the assumption it is based on, while true of
> a vanilla install, is NOT true in a customized install environment.
>
> I'd prefer to see every module live in its own subdirectory of the
> modules directory, much as is does in 4.x, but more rigorously (ie,
> separate Sales Orders from Purchase Orders, don't group them in Orders)
> and each module should have its own, atomic codebase.
>
> We're doing things in the field that are far more advanced than just
> custom fields (although we make use of those too) and we need the
> flexibility that comes with exposing the dirty details.
>
> Same thing with database queries. It's tempting I know to try and
> abstract all those away with wrapper functions (and there are some
> wrapper functions that are OK - something like
> getContactEmail($contactID) or getContactAccount($contactID) or
> isDeleted($crmID) are all simple and handy and useful for simplifying
> out some of the database access overhead.
>
> But as soon as you start getting into more complex queries, I want to
> see the SQL right there in the code. I need to understand EXACTLY what
> is going on, and I may need to muck with the query; especially if I have
> changed/extended the database/table.
>
> For example, our users had a need for Quotes to preserve the order in
> which Products were added to the Quote. The intent is that they wanted
> the PDF export of a Quote to do something like this:
>
> Big expensive item 1
> Option A for item 1
> Option B for item 1
> Option C for item 1
> Big expensive item 2
> Option A for item 2
> Option C for item 2
>
> But as the items came out of the database in the order in which they
> were added to the database, what they got once they saved it might be
> something like:
>
> Option C for Item 2
> Option A for item 1
> Big Expensive item 2
> ...etc
>
> >From a developer point of view, that's OK, 'cause all the items are on
> there and the total is right, so what's the big deal? Well to them, this
> WAS a big deal - a really big deal. They wanted this changed, and my
> boss wanted it to happen NOW; so I extended the QuoteProductsRel to
> include a sequenceNumber.
>
> Then they wanted to be able to add headings, like this:
>
> Heading 1
> Big expensive item 1
> Option A for item 1
> Option B for item 1
> Option C for item 1
>
> Heading 2
> Big expensive item 2
> Option A for item 2
> Option C for item 2
>
> So I extended QuoteProductsRel to include a heading field that, if
> populated, would print on the line preceding the product it was
> associated with.
>
> Etc etc etc.
>
> This sort of stuff is WAY easier if the code to do it is right there,
> and not buried in a common function somewhere (like a lot of the UI code
> is, where you have the Mother of all If Statements lurking in utils.php
> that has to account for *every single module* before it presents the UI.
>
> I'd like to see some standardization on the module file structure
> though:
>
> modules/Foo/EditView.php for the edit UI for the module object
> modules/Foo/DetailView.php for the detailed view of the object
> modules/Foo/Save.php for saving the object to the DB
> modules/Foo/CreatePDF.php for creating a PDF version of the object
> modules/Foo/UIElements.php for the common UI functions (that are now
> stuffed in include/utils.php
> modules/Foo/BarPopup.php if there is a popup window to select associated
> Bar objects
>
> etc.
>
> It makes dealing with modules easier if we know where the various
> functions are kept.
>
> DG
>
> _______________________________________________
> Get started with creating presentations online - http://zohoshow.com?vt
>
_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Ivar Simpkins
In reply to this post by Dennis Grant


Sorry DG this doesn’t wear with me.

'The price of programmability' is what it's about.

 There is the short term solution and it has its merits, but planning a
structured API can only have its long term benefits. Where does vTiger
want to be in the future, this has to be answered. If it wants to be in
the top ratings, then I think every effort has to be made, to
reconsolidate and then plan for the future.

Sure the learning curve is steep but the rewards are enormous. Would it
not be better to prefect a generic function, in an OO structure which
does all you want, one entrance, one exit and away you go.

A well documented code base that automatically builds your API
documentation gives you a valuable toolbox, now concentrate the
communities efforts on pulling together and raising the lowest common
denominator and you have a work the progresses moving a an incredible
rate.

And as Matt said:

'What do you think?  Am I nuts?  Does it take too much away from the
developer in the name of standardization?  To me it feels like the next
natural step to cleaning up the system and creating defined layers as
far as security, display and entities are concerned.  It should cut down
on all the unnecessary code re-creation in the system as well.'

NO, you are not nuts, it gives you the long term ability to use ones
creativity to best avail without (always) having to worry about the nuts
& bolts.

I think this is a courageous step in the right direction, where, after
surviving the learning curve, all will benefit, creating a homogeneous
standardised kernel, with the ability of Plug 'n Play ala Joomla and
probably much more.

'The price of programmability' is high but seeing the caliber of work
produced so far, I am convinced Matt is on the right track to creating a
new dimension in CRM.

Please excuse the tone but its hard to see the same mistake been made
over and again.

Ivar

> My basic idea comes down to writing wrapper classes that are used to
> abstract most of the unnecessary functions from developers and instead
> provides a uniform API to access modules, permissions, input, etc.

Oooh.

> The idea comes in three pieces.  Common UI classes, most of which is
> already done with the new UI.  A "mainframe" engine similar to the
> joomla $mainframe.  And last but certainly not least is an entity
engine
> that would abstract most of the entity details from developers and
truly
> provide the unified API.

To be honest, I think this is moving in the wrong direction.

If everybody used the same vanilla VTiger installation, this would make
a lot of sense - one common UI for the entire set of users, and a much
cleaner codebase for developers. Happy happy for everybody.

Unfortunately, a lot of us are NOT using vanilla VTiger; we have to
customize the application to meet the needs of our user base. And a lot
of the time, our users really really want changes that may seem utterly
trivial from a developer standpoint, but for whatever reason, they
REALLY want it that way.

And the customer is always right, so we have to do it.

I don't, for example, want to see something like where Accounts and
Contacts share the same DetailView module - because in my setup, there
are large differences between the DetailView we do for Accounts, and the
DetailView we do for Contacts.

It is a whole lot easier to deal with each module having its own code,
even if that code is 95% common with other modules in a vanilla install,
because that way, if I'm working on the Contacts DetailView, I don't
have to worry about changes spilling over into the operation of another
module.

Yes, it makes changing common code more difficult, 'cause now I have to
drill down into each submodule and make the same change over and over
again, and I can certainly see where modularizing and sharing code makes
that job a lot easier. But the assumption it is based on, while true of
a vanilla install, is NOT true in a customized install environment.

I'd prefer to see every module live in its own subdirectory of the
modules directory, much as is does in 4.x, but more rigorously (ie,
separate Sales Orders from Purchase Orders, don't group them in Orders)
and each module should have its own, atomic codebase.
 
We're doing things in the field that are far more advanced than just
custom fields (although we make use of those too) and we need the
flexibility that comes with exposing the dirty details.

Same thing with database queries. It's tempting I know to try and
abstract all those away with wrapper functions (and there are some
wrapper functions that are OK - something like
getContactEmail($contactID) or getContactAccount($contactID) or
isDeleted($crmID) are all simple and handy and useful for simplifying
out some of the database access overhead.

But as soon as you start getting into more complex queries, I want to
see the SQL right there in the code. I need to understand EXACTLY what
is going on, and I may need to muck with the query; especially if I have
changed/extended the database/table.

For example, our users had a need for Quotes to preserve the order in
which Products were added to the Quote. The intent is that they wanted
the PDF export of a Quote to do something like this:

Big expensive item 1
Option A for item 1
Option B for item 1
Option C for item 1
Big expensive item 2
Option A for item 2
Option C for item 2

But as the items came out of the database in the order in which they
were added to the database, what they got once they saved it might be
something like:

Option C for Item 2
Option A for item 1
Big Expensive item 2
...etc

>From a developer point of view, that's OK, 'cause all the items are on
there and the total is right, so what's the big deal? Well to them, this
WAS a big deal - a really big deal. They wanted this changed, and my
boss wanted it to happen NOW; so I extended the QuoteProductsRel to
include a sequenceNumber.

Then they wanted to be able to add headings, like this:

Heading 1
Big expensive item 1
Option A for item 1
Option B for item 1
Option C for item 1

Heading 2
Big expensive item 2
Option A for item 2
Option C for item 2

So I extended QuoteProductsRel to include a heading field that, if
populated, would print on the line preceding the product it was
associated with.

Etc etc etc.

This sort of stuff is WAY easier if the code to do it is right there,
and not buried in a common function somewhere (like a lot of the UI code
is, where you have the Mother of all If Statements lurking in utils.php
that has to account for *every single module* before it presents the UI.

I'd like to see some standardization on the module file structure
though:

modules/Foo/EditView.php for the edit UI for the module object
modules/Foo/DetailView.php for the detailed view of the object
modules/Foo/Save.php for saving the object to the DB
modules/Foo/CreatePDF.php for creating a PDF version of the object
modules/Foo/UIElements.php for the common UI functions (that are now
stuffed in include/utils.php
modules/Foo/BarPopup.php if there is a popup window to select associated
Bar objects

etc.

It makes dealing with modules easier if we know where the various
functions are kept.

DG

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 


--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.10.0/388 - Release Date:
2006/07/13
 

--
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.10.0/388 - Release Date:
2006/07/13
 


_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Matthew Brichacek
In reply to this post by Allan Bush
On Fri, 2006-07-14 at 07:32 -0700, Allan Bush wrote:

> Proper use of a OO design would solves this for both of you.
>
> All the common code and default module behavior should be moved into
> the a set of default classes or "engines" as Matthew is calling them.
> Then for each module we inherit the classes and make a couple method
> calls (ideally this is abstracted as well so to create a module all
> you do is create a class which inherits the base class) which
> generates the required database actions / html page.  If you want
> something different then the default, like Dennis describes, you
> simply extent the required method in your inherited class.  This way
> the common code is abstracted and only needs to exist once while the
> module specific code is in the module for easy modification.
I think we're right on the same page here, but if you take for example
the Campaign.php class, it should not be needed. All of the functions
and variables it defines are generic and do not need to be re-created
for each module that does simple data collection and display.
If you want to write a module that does more than this, and will need
it's own class, by all means, extend the EntityEngine class (or
CRMEntity if it becomes the engine) and then continue on your way.

>
> The code is already partly structured this way (it's just not used
> properly).  There's already a base class CRMentity which every module
> inherits from.  What needs to be done is to clean up the CRMentity -
> module relationship so that only methods common to all modules and
> their default behavior is defined in CRMentity and only things which
> need to be changed are defined in the module class.  
But even things that need to be changed, may not need to be changed and
defined in separate module classes.  For example all the get_leads,
get_contacts, etc functions can be standardized and instead use a set of
DB keys to track relationship_types allowed for a module.
function get_leads($id,$entity) {
        if($this->is_relationship_type($entity->module,"Leads")) {
                stuff..
        } else
                return false;
}

> Then we need to
> create a class for each page type (again some of this already exists.
> ala inculdes/Listview.php, but here it isn't used at all) with the
> default behaviors of each page type.  Now we have an option, either we
> have each module inherit these page type classes (making changes as
> needed ala CRMentity) and have a common set of calls to render each
> page type or we create a SMALL file in each module with a couple of
> calls to the page type class to render the page and we substitute new
> functions as needed for customizations.  Personally I think the first
> option here is the better design, but unless you get it right it could
> become difficult to modify the parts of the page you want without
> re-witting a lot of the base class (should be avoided at all cost or
> we're right back in the mess we are in now).
Correct, and I think if you created the correct UI classes you could
very easily overwrite the functions you want to customize and _only_ the
functions you want to customize (like AddBlock() for example).


> That was a bit of a long winded explanation, let me know if anyone
> requires clarification.
I think we're pretty close to being in the same school of though but
maybe I needed to explain my thoughts in a bit more detail.  I really
just laid out a very high level overview of what I think should be done.

Matt

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Matthew Brichacek
In reply to this post by Dennis Grant

> I don't, for example, want to see something like where Accounts and
> Contacts share the same DetailView module - because in my setup, there
> are large differences between the DetailView we do for Accounts, and the
> DetailView we do for Contacts.
So in these cases you could over-ride the system DetailView by creating
<vtiger_root>/Accounts/DetailView.html.php for your actual HTML stuff,
and <vtiger_root>/Accounts/Detailview.php if you want to change the
logic flow.   If you are only changing the look/feel, then re-declare
the AddBlock() function with your new special sauce inside of it and off
you go.


> It is a whole lot easier to deal with each module having its own code,
> even if that code is 95% common with other modules in a vanilla install,
> because that way, if I'm working on the Contacts DetailView, I don't
> have to worry about changes spilling over into the operation of another
> module.
See above, the correct separation of these layers in the system can
still give you the flexibility you need.


> We're doing things in the field that are far more advanced than just
> custom fields (although we make use of those too) and we need the
> flexibility that comes with exposing the dirty details.
>
To create new field types it would just be a matter of creating a new
UIType, populating the DB fields you want to have that uitype (via XML),
and then creating the base HTML for it in the UIType.tpl file.
{if {$uitype} == "230"}
your_new_html
{/if}
On top of this you could just over-ride the system UI creator and
replace it with your own display classes.

> Same thing with database queries. It's tempting I know to try and
> abstract all those away with wrapper functions (and there are some
> wrapper functions that are OK - something like
> getContactEmail($contactID) or getContactAccount($contactID) or
> isDeleted($crmID) are all simple and handy and useful for simplifying
> out some of the database access overhead.
>
> But as soon as you start getting into more complex queries, I want to
> see the SQL right there in the code. I need to understand EXACTLY what
> is going on, and I may need to muck with the query; especially if I have
> changed/extended the database/table.
I actually think the entity engine will need to have a query builder in
it so that the queries can be built dynamically depending on the type of
data you are going for.  A true entity engine (like the one in
ofbiz.org) will not ever let you see a query, nor should you need to if
the entity engine is doing it's job correctly.

>
> For example, our users had a need for Quotes to preserve the order in
> which Products were added to the Quote. The intent is that they wanted
> the PDF export of a Quote to do something like this:
>
> Big expensive item 1
> Option A for item 1
> Option B for item 1
> Option C for item 1
> Big expensive item 2
> Option A for item 2
> Option C for item 2
>
> But as the items came out of the database in the order in which they
> were added to the database, what they got once they saved it might be
> something like:
>
> Option C for Item 2
> Option A for item 1
> Big Expensive item 2
> ...etc
>
> >From a developer point of view, that's OK, 'cause all the items are on
> there and the total is right, so what's the big deal? Well to them, this
> WAS a big deal - a really big deal. They wanted this changed, and my
> boss wanted it to happen NOW; so I extended the QuoteProductsRel to
> include a sequenceNumber.
>
> Then they wanted to be able to add headings, like this:
>
> Heading 1
> Big expensive item 1
> Option A for item 1
> Option B for item 1
> Option C for item 1
>
> Heading 2
> Big expensive item 2
> Option A for item 2
> Option C for item 2
>
> So I extended QuoteProductsRel to include a heading field that, if
> populated, would print on the line preceding the product it was
> associated with.
>
> Etc etc etc.
>
> This sort of stuff is WAY easier if the code to do it is right there,
> and not buried in a common function somewhere (like a lot of the UI code
> is, where you have the Mother of all If Statements lurking in utils.php
> that has to account for *every single module* before it presents the UI.
i hate utils.php :).  But the examples you lay out above would not at
all be impossible.  You would simply tell the entity engine (via an XML
file) that you want a seqno and header value assigned to each entity of
type=Quotes, and then in the query builder it will check for seqno and
if it find it, ORDER BY seqno will be appended to the query string.


Really what I am trying to accomplish is to make your life easier in the
long run than what it currently is, even when diverging from the
standard way of doing it with-in vtiger.  There is a learning curve that
will come along with this but correct documentation and developer
support via this mailing list should help you get past any of that.

Matt


_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Allan Bush
In reply to this post by Matthew Brichacek
I'm pretty sure we're on the same page Matt, my reply was more of an
attempt to convince Dennis then an attempt to disagree with anything
you said.

In your example though I'm not sure how much I like having a function
called "get_leads" in the base class.  I think there's way too much
coupling between "modules" right now and I'd like to get away from
that, however I'm not sure what the best way to do that is without
losing any of the control we currently have.  That's probably another
discussion though.

I agree with your design, I could probably argue all day over the
implementation of the details, but as long as it's constant and
logical I don't care too much.


On 7/14/06, Matthew Brichacek <[hidden email]> wrote:

> On Fri, 2006-07-14 at 07:32 -0700, Allan Bush wrote:
> > Proper use of a OO design would solves this for both of you.
> >
> > All the common code and default module behavior should be moved into
> > the a set of default classes or "engines" as Matthew is calling them.
> > Then for each module we inherit the classes and make a couple method
> > calls (ideally this is abstracted as well so to create a module all
> > you do is create a class which inherits the base class) which
> > generates the required database actions / html page.  If you want
> > something different then the default, like Dennis describes, you
> > simply extent the required method in your inherited class.  This way
> > the common code is abstracted and only needs to exist once while the
> > module specific code is in the module for easy modification.
> I think we're right on the same page here, but if you take for example
> the Campaign.php class, it should not be needed. All of the functions
> and variables it defines are generic and do not need to be re-created
> for each module that does simple data collection and display.
> If you want to write a module that does more than this, and will need
> it's own class, by all means, extend the EntityEngine class (or
> CRMEntity if it becomes the engine) and then continue on your way.
>
> >
> > The code is already partly structured this way (it's just not used
> > properly).  There's already a base class CRMentity which every module
> > inherits from.  What needs to be done is to clean up the CRMentity -
> > module relationship so that only methods common to all modules and
> > their default behavior is defined in CRMentity and only things which
> > need to be changed are defined in the module class.
> But even things that need to be changed, may not need to be changed and
> defined in separate module classes.  For example all the get_leads,
> get_contacts, etc functions can be standardized and instead use a set of
> DB keys to track relationship_types allowed for a module.
> function get_leads($id,$entity) {
>         if($this->is_relationship_type($entity->module,"Leads")) {
>                 stuff..
>         } else
>                 return false;
> }
>
> > Then we need to
> > create a class for each page type (again some of this already exists.
> > ala inculdes/Listview.php, but here it isn't used at all) with the
> > default behaviors of each page type.  Now we have an option, either we
> > have each module inherit these page type classes (making changes as
> > needed ala CRMentity) and have a common set of calls to render each
> > page type or we create a SMALL file in each module with a couple of
> > calls to the page type class to render the page and we substitute new
> > functions as needed for customizations.  Personally I think the first
> > option here is the better design, but unless you get it right it could
> > become difficult to modify the parts of the page you want without
> > re-witting a lot of the base class (should be avoided at all cost or
> > we're right back in the mess we are in now).
> Correct, and I think if you created the correct UI classes you could
> very easily overwrite the functions you want to customize and _only_ the
> functions you want to customize (like AddBlock() for example).
>
>
> > That was a bit of a long winded explanation, let me know if anyone
> > requires clarification.
> I think we're pretty close to being in the same school of though but
> maybe I needed to explain my thoughts in a bit more detail.  I really
> just laid out a very high level overview of what I think should be done.
>
> Matt
>
> _______________________________________________
> Get started with creating presentations online - http://zohoshow.com?vt
>
_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Matthew Brichacek
On Fri, 2006-07-14 at 10:13 -0700, Allan Bush wrote:

> I'm pretty sure we're on the same page Matt, my reply was more of an
> attempt to convince Dennis then an attempt to disagree with anything
> you said.
>
> In your example though I'm not sure how much I like having a function
> called "get_leads" in the base class.  I think there's way too much
> coupling between "modules" right now and I'd like to get away from
> that, however I'm not sure what the best way to do that is without
> losing any of the control we currently have.  That's probably another
> discussion though.
I'm not a big fan of how this is done either but you're right, this
falls into another discussion about how to clean up current practices
going forward with a new structure.

>
> I agree with your design, I could probably argue all day over the
> implementation of the details, but as long as it's constant and
> logical I don't care too much.
Plenty of details that are yet to be worked out ;).



Matt

>
>
> On 7/14/06, Matthew Brichacek <[hidden email]> wrote:
> > On Fri, 2006-07-14 at 07:32 -0700, Allan Bush wrote:
> > > Proper use of a OO design would solves this for both of you.
> > >
> > > All the common code and default module behavior should be moved into
> > > the a set of default classes or "engines" as Matthew is calling them.
> > > Then for each module we inherit the classes and make a couple method
> > > calls (ideally this is abstracted as well so to create a module all
> > > you do is create a class which inherits the base class) which
> > > generates the required database actions / html page.  If you want
> > > something different then the default, like Dennis describes, you
> > > simply extent the required method in your inherited class.  This way
> > > the common code is abstracted and only needs to exist once while the
> > > module specific code is in the module for easy modification.
> > I think we're right on the same page here, but if you take for example
> > the Campaign.php class, it should not be needed. All of the functions
> > and variables it defines are generic and do not need to be re-created
> > for each module that does simple data collection and display.
> > If you want to write a module that does more than this, and will need
> > it's own class, by all means, extend the EntityEngine class (or
> > CRMEntity if it becomes the engine) and then continue on your way.
> >
> > >
> > > The code is already partly structured this way (it's just not used
> > > properly).  There's already a base class CRMentity which every module
> > > inherits from.  What needs to be done is to clean up the CRMentity -
> > > module relationship so that only methods common to all modules and
> > > their default behavior is defined in CRMentity and only things which
> > > need to be changed are defined in the module class.
> > But even things that need to be changed, may not need to be changed and
> > defined in separate module classes.  For example all the get_leads,
> > get_contacts, etc functions can be standardized and instead use a set of
> > DB keys to track relationship_types allowed for a module.
> > function get_leads($id,$entity) {
> >         if($this->is_relationship_type($entity->module,"Leads")) {
> >                 stuff..
> >         } else
> >                 return false;
> > }
> >
> > > Then we need to
> > > create a class for each page type (again some of this already exists.
> > > ala inculdes/Listview.php, but here it isn't used at all) with the
> > > default behaviors of each page type.  Now we have an option, either we
> > > have each module inherit these page type classes (making changes as
> > > needed ala CRMentity) and have a common set of calls to render each
> > > page type or we create a SMALL file in each module with a couple of
> > > calls to the page type class to render the page and we substitute new
> > > functions as needed for customizations.  Personally I think the first
> > > option here is the better design, but unless you get it right it could
> > > become difficult to modify the parts of the page you want without
> > > re-witting a lot of the base class (should be avoided at all cost or
> > > we're right back in the mess we are in now).
> > Correct, and I think if you created the correct UI classes you could
> > very easily overwrite the functions you want to customize and _only_ the
> > functions you want to customize (like AddBlock() for example).
> >
> >
> > > That was a bit of a long winded explanation, let me know if anyone
> > > requires clarification.
> > I think we're pretty close to being in the same school of though but
> > maybe I needed to explain my thoughts in a bit more detail.  I really
> > just laid out a very high level overview of what I think should be done.
> >
> > Matt
> >
> > _______________________________________________
> > Get started with creating presentations online - http://zohoshow.com?vt
> >
> _______________________________________________
> Get started with creating presentations online - http://zohoshow.com?vt 

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Dennis Grant
In reply to this post by Allan Bush

> Really what I am trying to accomplish is to make your life easier in
the
> long run than what it currently is, even when diverging from the
> standard way of doing it with-in vtiger.  There is a learning curve
that
> will come along with this but correct documentation and developer
> support via this mailing list should help you get past any of that.

Except that you're killing me with your good intentions.

The life of an integrator doesn't permit steep learning curves and heavy
abstractions; there just isn't time to do so. I've got a dozen different
systems to get to play together, a boss screaming at me to get this
stuff done yesterday, and no time to devote to learning whatever
abstraction layer you've decided to implement.

And Murphy's Law being what it is, if your abstraction model has one
flaw, that will be the thing that my boss wants as his top priority.

"Perfect is the enemy of good enough" I don't want "perfect"; I want
"good enough" I want to be able to dive into the source and start
modifying how the thing works *right now* without having to trace back a
billion functions or wrap my brain around your object model. I don't
have time to do so, because I'm simultaneously trying to play with a
dozen other pieces of the puzzle.

Guys, I have been playing this game for a LONG time. I have written
major applications to perform mission-critical business functions.(At
one time, if my code broke DaimlerChrysler stopped building cars) I have
seen and dealt with thousands of software projects, both as a
participant and as an observer, and almost without fail, every time
somebody sets out to do the "perfect" object-based, extensible,
customizable framework, it just leads to tears.

I am utterly convinced that the way to success is to write code that is
optimised for human legibility and comprehension. If you write code that
any decent developer can pick up, find his bearings in a couple of
seconds, and then comprehend *without* the benefit of having read the
thousand-page project definition in advance, then you are doing well.

The second I have to go to a developer list to get assistance you, as a
developer, have FAILED ME as an integrator - because if your code is so
illegible (or so heavily abstracted, or has such a steep learning curve,
or whatever) that I cannot figure it out on my own in a few minutes and
so have to drop everything until I can get some assistance... well then,
you've burned up my precious, precious time.

That code might be sheer genius. It might be, once you have tackled the
learning curve, breathtakingly beautiful. But if I can't just fire up vi
and dive in, then it is WRONG.

Do you see what I'm getting at? Keep it SIMPLE and you win. Overthink
it, and you'll wind up with something that only YOU understand... and
I'm back with my privately-maintained fork.

The big problem with the current state of Vtiger 4.x is that it
straddles both worlds. There are parts of the code that are fairly
straightforward and out in plain sight, and only lack comments to make
them "good enough". Then there are parts that want to be object-oriented
with an API, but really depend on a couple of heavily-overloaded
megafunctions buried deep in the depths of util.php (also uncommented)
so you have to play API detective to figure out just what the hell is
happening where. And then there's parts of the code that mix both.

It'd be SO much easier to maintain if all the code that related to a
module (less some *basic* toolkit functions that could reside in a
common include) just lived in that module.

Please please PLEASE, for the love of GOD, don't go down the path of
objectifying everything and abstracting the code to the point of
incomprehensibility.

DG  

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Dan Means
I couldn't have said this better than Dennis....

--
Dan Means
Mission Viejo, CA
www.dkmeansonline.com
www.teamsrs.com

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Caryl Takvorian
In reply to this post by Dennis Grant
Just for the sake of it, let me say that I whole heartedly agree with Dennis.
I'm not unfortunately convinced that his (our) opinion will change anything, but I find the arguments truthfully compelling.

Caryl



On 7/14/06, Dennis Grant <[hidden email]> wrote:

> Really what I am trying to accomplish is to make your life easier in
the
> long run than what it currently is, even when diverging from the
> standard way of doing it with-in vtiger.  There is a learning curve
that
> will come along with this but correct documentation and developer
> support via this mailing list should help you get past any of that.

Except that you're killing me with your good intentions.

The life of an integrator doesn't permit steep learning curves and heavy
abstractions; there just isn't time to do so. I've got a dozen different
systems to get to play together, a boss screaming at me to get this
stuff done yesterday, and no time to devote to learning whatever
abstraction layer you've decided to implement.

And Murphy's Law being what it is, if your abstraction model has one
flaw, that will be the thing that my boss wants as his top priority.

"Perfect is the enemy of good enough" I don't want "perfect"; I want
"good enough" I want to be able to dive into the source and start
modifying how the thing works *right now* without having to trace back a
billion functions or wrap my brain around your object model. I don't
have time to do so, because I'm simultaneously trying to play with a
dozen other pieces of the puzzle.

Guys, I have been playing this game for a LONG time. I have written
major applications to perform mission-critical business functions.(At
one time, if my code broke DaimlerChrysler stopped building cars) I have
seen and dealt with thousands of software projects, both as a
participant and as an observer, and almost without fail, every time
somebody sets out to do the "perfect" object-based, extensible,
customizable framework, it just leads to tears.

I am utterly convinced that the way to success is to write code that is
optimised for human legibility and comprehension. If you write code that
any decent developer can pick up, find his bearings in a couple of
seconds, and then comprehend *without* the benefit of having read the
thousand-page project definition in advance, then you are doing well.

The second I have to go to a developer list to get assistance you, as a
developer, have FAILED ME as an integrator - because if your code is so
illegible (or so heavily abstracted, or has such a steep learning curve,
or whatever) that I cannot figure it out on my own in a few minutes and
so have to drop everything until I can get some assistance... well then,
you've burned up my precious, precious time.

That code might be sheer genius. It might be, once you have tackled the
learning curve, breathtakingly beautiful. But if I can't just fire up vi
and dive in, then it is WRONG.

Do you see what I'm getting at? Keep it SIMPLE and you win. Overthink
it, and you'll wind up with something that only YOU understand... and
I'm back with my privately-maintained fork.

The big problem with the current state of Vtiger 4.x is that it
straddles both worlds. There are parts of the code that are fairly
straightforward and out in plain sight, and only lack comments to make
them "good enough". Then there are parts that want to be object-oriented
with an API, but really depend on a couple of heavily-overloaded
megafunctions buried deep in the depths of util.php (also uncommented)
so you have to play API detective to figure out just what the hell is
happening where. And then there's parts of the code that mix both.

It'd be SO much easier to maintain if all the code that related to a
module (less some *basic* toolkit functions that could reside in a
common include) just lived in that module.

Please please PLEASE, for the love of GOD, don't go down the path of
objectifying everything and abstracting the code to the point of
incomprehensibility.

DG

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt


_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt 
Reply | Threaded
Open this post in threaded view
|

Re: Ideas for vtiger architecture moving forward

Matthew Brichacek
In reply to this post by Dennis Grant
OK, so I am going to highlight and comment on some of my favorite quotes
from DG so far, this way when I start ignoring his emails you will all
understand why.

1) It is a whole lot easier to deal with each module having its own
code, even if that code is 95% common with other modules.

This has to be hands-down the stupidest thing I have ever heard in my
life and I can't believe that anyone who claims the experience you do
actually posts this drivel on a public mailing list.

2) I am utterly convinced that the way to success is to write code that
is optimised for human legibility and comprehension.

I am utterly convinced that anyone who argues that 95% code similarities
in vtiger modules is a "good enough" method is utterly stupid.

3) The second I have to go to a developer list to get assistance you, as
a developer, have FAILED ME as an integrator ... well then, you've
burned up my precious, precious time

So your 'job' as an OSS systems integrator has never included needing to
RTFM or actually expand your horizons or even.. *GASP* ask a fsck'ing
question on a mailing list?

4) That code might be sheer genius. But if I can't just fire up vi
and dive in, then it is WRONG.

I'm starting to see the picture now.. you're lazy, and rather than do
something about the current problems in the system you are going to
whine about it.  Sergio was right, my apologies Sergio.

5) Do you see what I'm getting at? Keep it SIMPLE and you win. Overthink
it, and you'll wind up with something that only YOU understand... and
I'm back with my privately-maintained fork.

I think I've made it clear that simple, concise and logical are all
design goals of the system.  And you sir have made it clear that you
have no intention of helping move this project forward by displaying
your willingness to fork instead of work with a _real_ design structure
that a _real_ OSS systems integrator would be very happy to have.

Matt

On Fri, 2006-07-14 at 15:37 -0400, Dennis Grant wrote:

> > Really what I am trying to accomplish is to make your life easier in
> the
> > long run than what it currently is, even when diverging from the
> > standard way of doing it with-in vtiger.  There is a learning curve
> that
> > will come along with this but correct documentation and developer
> > support via this mailing list should help you get past any of that.
>
> Except that you're killing me with your good intentions.
>
> The life of an integrator doesn't permit steep learning curves and heavy
> abstractions; there just isn't time to do so. I've got a dozen different
> systems to get to play together, a boss screaming at me to get this
> stuff done yesterday, and no time to devote to learning whatever
> abstraction layer you've decided to implement.
>
> And Murphy's Law being what it is, if your abstraction model has one
> flaw, that will be the thing that my boss wants as his top priority.
>
> "Perfect is the enemy of good enough" I don't want "perfect"; I want
> "good enough" I want to be able to dive into the source and start
> modifying how the thing works *right now* without having to trace back a
> billion functions or wrap my brain around your object model. I don't
> have time to do so, because I'm simultaneously trying to play with a
> dozen other pieces of the puzzle.
>
> Guys, I have been playing this game for a LONG time. I have written
> major applications to perform mission-critical business functions.(At
> one time, if my code broke DaimlerChrysler stopped building cars) I have
> seen and dealt with thousands of software projects, both as a
> participant and as an observer, and almost without fail, every time
> somebody sets out to do the "perfect" object-based, extensible,
> customizable framework, it just leads to tears.
>
> I am utterly convinced that the way to success is to write code that is
> optimised for human legibility and comprehension. If you write code that
> any decent developer can pick up, find his bearings in a couple of
> seconds, and then comprehend *without* the benefit of having read the
> thousand-page project definition in advance, then you are doing well.
>
> The second I have to go to a developer list to get assistance you, as a
> developer, have FAILED ME as an integrator - because if your code is so
> illegible (or so heavily abstracted, or has such a steep learning curve,
> or whatever) that I cannot figure it out on my own in a few minutes and
> so have to drop everything until I can get some assistance... well then,
> you've burned up my precious, precious time.
>
> That code might be sheer genius. It might be, once you have tackled the
> learning curve, breathtakingly beautiful. But if I can't just fire up vi
> and dive in, then it is WRONG.
>
> Do you see what I'm getting at? Keep it SIMPLE and you win. Overthink
> it, and you'll wind up with something that only YOU understand... and
> I'm back with my privately-maintained fork.
>
> The big problem with the current state of Vtiger 4.x is that it
> straddles both worlds. There are parts of the code that are fairly
> straightforward and out in plain sight, and only lack comments to make
> them "good enough". Then there are parts that want to be object-oriented
> with an API, but really depend on a couple of heavily-overloaded
> megafunctions buried deep in the depths of util.php (also uncommented)
> so you have to play API detective to figure out just what the hell is
> happening where. And then there's parts of the code that mix both.
>
> It'd be SO much easier to maintain if all the code that related to a
> module (less some *basic* toolkit functions that could reside in a
> common include) just lived in that module.
>
> Please please PLEASE, for the love of GOD, don't go down the path of
> objectifying everything and abstracting the code to the point of
> incomprehensibility.
>
> DG  
>
> _______________________________________________
> Get started with creating presentations online - http://zohoshow.com?vt 

_______________________________________________
Get started with creating presentations online - http://zohoshow.com?vt