I've been doing a lot of work with modules in Mach-II in the past couple months and thought I'd blog about some of the stumbling blocks I've encountered.

First, a little background:

A Mach-II module is an application which runs inside of a parent Mach-II application. A module usually groups together common functionality in to a related package, encapsulates that package nicely within your Mach-II app, and inherits a lot of the base configuration from the parent Mach-II application. Some people build Mach-II modules that are meant to be reused across multiple applications, for multiple sites, while others build open-source modules which are meant to be redistributable and used by anyone, anywhere. My situation is a little different.

I'm in the process of re-architecting one of our major applications, getting years of old cruff and rebuilding everything on top of Mach-II 1.8. A large part of this project entails taking some existing Mach-II apps (build under 1.1, 1.5 and 1.6) and integrating them in to the new, main Mach-II 1.8 app. A good amount of the work involves throwing out what's handled by the parent app, adding in new parent-child integration points where needed, and refactoring views to get them to work inside of modules.

So that's the context for my work. Your experience may be different, of course, especially if you are looking to build a standalone module that can be integrated in to anyone's Mach-II app, like the super-fantastic Mach-II Dashboard application or Mach Blog. (BTW: If you develop with Mach-II and you're not using the Mach-II Dashboard application, you're wasting time and effort. Single-listener, plugin, filter or config reload is a huge timesaver — and that's only part of what you can do with the Dashboard.)

The full introduction to Mach-II modules can be found on the Mach-II wiki. I'm going to assume that you're quite familiar with how Mach-II works and know your way around a Mach-II configuration file.

I also want to say that I've not used each and every feature supported by Mach-II modules. I've not had need (yet) for things like the runParent attribute of the <plugin> tag. If you're interested in that topic, please see this thread on the always-useful Mach-II Google group.

There's a lot of ground to cover, so I thought I'd break things apart in to a series of posts:

• Modules and Mach-II config files
• Modules and ColdSpring
• Modules and Views
• Converting an Existing App to a Module

I'm going to try to get one of these out a week. In the end, if Team Mach-II finds it interesting, useful, and remotely accurate, I'll consolidate some of this to post on the Mach-II wiki.

I've been going through Daniel Short's excellent new training module, "ColdFusion 9 New Features" over on Lynda.com. I really like the depth and breadth of content that's available on Lynda.com (and it's not expensive), and given that Adobe's on in-person training for CF9 won't be available for a while yet, it's great to see them rolling out CF9 training at the same time as the CF9 release.

I've been playing with CF9 for quite a while now, and Daniel does a great job at covering all the major new features, and some minor ones. He covers one cool, very useful thing that I wasn't aware of in CF9: the ability to programmatically enable robust exception handling.

Let's say that you have a production application that's throwing errors. You probably are (should be) already logging most of the activity in the application, but if you need to see what specific variables are being sent to a specific view page that's generating an error, sometimes dumping things out via CF's robust exception handling is the fastest way to go. You may not be able to set up a remote debugging session via CFBuilder on to a production server to get this information. You also certainly don't want robust exception handling enabled by default for anyone who accesses your application, as that's a big security hole. You just need to be able to see this information temporarily so you can figure out exactly what's going wrong.

In CF9, you can enable robust exception handling for a specific IP address by including the following in your application.cfc's constructor (eg; the area where you put all the <cfset this.someVariable=someValue /> statements):

The this.debuggingIPAddresses value is just a list of the IP address(es) that are allowed to see the debugging output and robust exception information. Add your IP address to this list, and you can see the full set of debugging and exception information in your production application.

Now you probably want to go one step further and wrap the above in a <cfif> statement that checks to make sure you actually should be showing this, or some random URL variable that your dev team knows about to turn this on on a per-request basis. This way you can enable robust exception handling on a per-request basis, without having to reload the entire application.

Like many people who develop Web applications, I don't have a formal background in computer science. I've taught myself most of what I know, and I've been very fortunate to be the recipient of a lot of great knowledge from people much, much smarter than I. One of the areas in which my skills can be improved is with utilizing the full power of SQL. I'm no SQL expert, and while I've certainly read up on stored procedures, the variances of T-SQL (as I use MS SQL Server at work), and fun stuff like cursors (and how you should use them judiciously, at best), most of my ColdFusion queries revolve around four simple SQL statements: SELECT, INSERT, UPDATE and DELETE.

As I've been refactoring some existing code as part of the development of a new, modern codebase for one of our most important applications, I've been trying to ask myself "OK, if I'm making a series of queries, can't I just do that all in the database instead?" The answer, often, is yes.

Here's an example, and one that I think is quite common: You need to query the database to check on the existence or state or value in a particular record. Then, depending on the value returned from that first query, you either do an update or an insert in to the database, or nothing at all. I've covered the basics of the upsert (a conditional INSERT or UPDATE depending on the existence of a matching record in the database) in a previous post, but I'd like to take it a step further in this example.

To be more specific in this example, I track all logins to the site. However, if the user logs in more than once in 10 minutes, I don't need to track that, as it's considered to be part of the same session. If I were going old-school with my code, I'd probably do something like this:

• Query the database to get the last login time of the user.
• If there's a record, compare the last login time to the current time, and if it is greater than 10 minutes ago, perform an update in the database.
• If there's no record in the database, perform an insert in the database.

So that will work out to three different calls, and a bunch of conditional s.

There's a simpler way to do all of this in the database. It involves using (gasp!) an IF tag in your SQL. Not too hard to do, really, and it takes care of everything in one query, rather than a series.

The code is as follows. I've included a "result flag" so you know whether any data update (insert or update) was performed.

IF statements in T-SQL work a whole lot like s, the major difference being the IF...BEGIN...END structure. It's really easy to figure out and, yes, Virginia, you can do ELSEs as well.

So instead of making two or three in ColdFusion, we've done the decision making inside of the database itself. This cuts down on traffic from the CF server to the database, which is always a good thing. Are you going to get a huge performance increase by doing this? Not necessarily, but it helps to keep the logic together, and simple when possible.

It's no secret that ColdFusion 9 will have Object-Relational Mapping capabilities baked in, thanks to the inclusion of the open-source and very powerful Hibernate ORM engine. In order to get ready for this major new addition to ColdFusion, I thought it would be wise to read up on Hibernate and take a look at what it can and can't do in CF9. CF9 is still very much a moving target, and we don't know for sure exactly what features in Hibernate it will support (though more information is leaking out), but it's good to go to the source and see what kind of possibilities might be available to us.

To that end, I picked up a copy of O'Reilly's "Harnessing Hibernate." It's a book targeted at Java developers. There's no mention of using Hibernate with CF, Groovy, Python or any other language in the book. So if you have absolutely no familiarity with Java, the significant time that the book spends walking through sample Java code is going to likely confuse you (although, really, Java is pretty easy to read if you write a lot of CFCs, write in cfscript or JavaScript). The book also assumes that you have at least a passing familiarity with Ant, and, by extension, Maven. It spends lots and lots of time on Maven, as a matter of fact, which I'll discuss a bit later. This should come as no surprise, really, because if you're doing a lot of Java work, you're going to have used Ant at least a few times.

The real strength of the book is in the examples it contains which demonstrate the advantages of using an ORM like Hibernate over managing object persistence on your own. The examples are very clear, and build upon each other throughout the book. All of the basics of using Hibernate are covered, in detail, including how Hibernate sessions work and the performance implications that Hibernate sessions have. A good chunk of the book is devoted to showing how easy Hibernate can make things for your Java projects, but as a ColdFusion developer, I'm used to my Web application middleware making things easier.

This also points to one of the shortcomings of the book: it spends so much time trying to convince you that Hibernate is great (and it is) and a huge time-saver (and it is), that once the basics are covered, and you're just at the point of wanting to go just beyond the basics, the authors refer you to the Hibernate documentation available online. I would have appreciated more examples of marking up the Hibernate.hbm.xml file to solve certain relational database and object mapping problems. There isn't a whole lot of XML on display here (and that's a good thing for some people), as Annotations are encouraged soon after the XML configuration is introduced. I suppose you could make the argument that the book shouldn't dwell too heavily on the XML config as that is documented on the Hibernate Web site, and that showing you how to change your code to utilize Hibernate is much more important.

As I hinted at before, the other major shortcoming of the book, to me, was the over-emphasis on Maven and utilizing Maven to manage build dependencies in your Java projects. The book could have been called "Harnessing Hibernate and Maven" for all the time spent on Maven throughout the book. Maven's great, I'm sure, but I wanted to learn primarily about Hibernate, not Maven.

The book has an excellent chapter on using Hibernate's Eclipse tools for modeling your Hibernate configuration, as well as a couple of chapters on Hibernate, Spring, and other Web frameworks. Those won't be of as immediate use to CF developers as Java developers, but, again, they help to provide a broad introduction to how Hibernate can be integrated in a variety of Java-based projects.

Having read quite a bit now about how you integrate Hiberante in to your Java projects and all the build and dependency resolution that needs to happen, I'm glad that I'll soon enough be able to work with Hibernate in CF, letting CF mask most of that setup complexity so I can spend my time on solving more interesting problems.

One of the things that I really like about using Mach-II, my MVC framework of choice for developing ColdFusion applications, is the level of support and interaction with the community that the core team behind the framework provides. I know it's a lot of work and a lot of time for them (and time is the one thing that none of us ever have enough of), and I'm always impressed with how they go out of their way to help boneheads like me in our hour of need.

One of the nifty features introduced in Mach-II 1.5 are property CFCs. In addition to supporting structs and arrays of properties in your Mach-II XML configuration file, you can also use CFCs if you want to do more advanced property setup. A common usage scenario for property CFCs is to store application globals. Prior to Mach-II 1.5, you'd create and store application globals in the configure() method of a plug-in, but that's really not what plug-ins in Mach-II are for. Plug-ins intercept execution points on each request, and weren't originally intended for application global functionality, but that's the hack that the community developed to work around a limitation of Mach-II.

When I first used property CFCs to set up application globals, I ran into a bit of a problem. My existing application globals were defined in an "appConstants" plug-in. The plug-in utilized objects managed via ColdSpring, because that's how I manage my object creation and dependency injection in CF (and, really, if you're not using ColdSpring, you really should). I needed to pull in a geographyService object from ColdSpring so I could put a query of states and query of countries in to my app globals. Pretty standard stuff.

Here's what I learned about doing this using Mach-II property CFCs:

1. Property CFCs are managed by Mach-II and not by ColdSpring. This is super-important and what really tripped me up.

Oftentimes in ColdSpring-managed objects where we're doing property injection (rather than constructor argument injection), we'll have something that looks like this (truncated for brevity):

This is pretty standard stuff for ColdSpring object injection. The trouble is that this doesn't really work for property CFCs in Mach-II 1.5 or later. Mach-II needs to handle the injection, not ColdSpring, so if you rely on ColdSpring to do this, you'll always have a variables.geographyService that = 0.

The solution? Use Mach-II 1.6's very handy "depends" autowiring enabled via Mach-II 1.6's ColdSpring property. If you're using Mach-II 1.6 or later you should be using the MachII.properties.ColdspringProperty instead of the older, and deprecated, ColdSpring plugin. The ColdSpringProperty in Mach-II 1.6 includes the ability to add the attribute "depends" to your <cfcomponent> declaration and it will autowire whatever objects you specify in to that object, without any additional calls or references to the ColdSpring framework. It's really sweet and very handy.

So for my appGloabls property CFC, my <cfcomponen> tag looks like this:

The depends="geographyService" attribute autowires the geographyService (managed by ColdSpring) in to this property CFC. Now I can call it and use it just as I did back in the days of my appConstants plug-in. For example:

2. If your property CFC depends on ColdSpring in any way, you need to declare that property CFC after the ColdSpring property declaration in your main Mach-II XML config file.

The always-helpful Peter J. Farrell (from the Mach-II team) explained to me that if you use <includes> in your Mach-II XML config file, as is the default example and method for utilizing the MachII.properties.ColdspringProperty, properties defined in the <includes> are parsed after the properties defined in your main Mach-II XML config file. So if you declare a property CFC that utilizes ColdSpring in your main Mach-II XML config file, it's going to get parsed and processed before the ColdSpring property gets parsed and set up, so the "depends" autowiring just won't work.

The solution? Make a really simple include for your property CFC or CFCs that use ColdSpring and declare that after your MachII.properties.ColdspringProperty include, as follows:

<includes>
<include file="config/coldspringProperty.xml" />
<include file="config/appGlobalsConfig.xml" />
</includes>

Utilizing this setup, I'm now able to utilize ColdSpring dependency injection and properly move my application globals setup in to a property CFC where it belongs.

When we create the first version of an application, we're prescient enough to always know the exact functional and storage needs of the application, no? I recently ran in to a bit of a problem caused by some poor planning on my part (or maybe it was the guy who wrote the original version of the app before me), and ran up against an unpleasant datatype conversion issue in SQL Server.

There's a field in the database for this particular app which stores relatively small numeric values — point values for each question in a quiz. This field is currently defined as a NUMERIC(4,2) in SQL Server. For some of you, this may have immediately raised a red flag. Why is this field limited to such a small range of numbers? Sometimes my long experience developing Web-based applications gets the better of me, and I concern myself with the limits of storage from an old-school "an 8GB hard drive costs $400!" point of view. And there are reasonable arguments to be made for limiting the size of fields within a database. However, my original reasoning was something like "Well they're not going to need values above 1000 nor decimal accuracy past two places, so why bother?"

You can see where this is going, right?

Now I have to support decimal accuracy to three places (and, sooner, or later, someone will ask me for four places) because in the world of scores for quizzes and exams, there's a big difference between 3.125 and 3.13.

So my problem was this: how to change the definition of the database field from NUMERIC(4,2) to a more standard NUMERIC(18,5)? Well you can't just fire up SQL Server Management Studio, right click the table, select "Design" and change the datatype. You'll get an internal conversion error from the database engine.

The next approach was to create a new column with the correct datatype and copy the contents of the old column in to the new one, drop the old column, and rename the new column with the old column name. In SQL, it's pretty trivial to copy one column to another. You just run:

However, when copying from a NUMERIC to a NUMERIC field, the database engine again throws the internal conversion error.

In this article from the SQL Server docs on CAST and CONVERT, there's a figure about 1/2 way down that shows which datatype conversions are supported in SQL Server. You'll notice that NUMERIC to NUMERIC is supported but it "Requires explicit CAST to prevent loss of precision or scale that might occur in an implicit conversion." So all I have to do is CAST() the origin field and we'll be OK, right?

Nope. Yet more internal conversion errors.

What I ended up doing was this, and I probably should have done everything in the database, but I was done wasting time and knew I could get this one time conversion done quickly using ColdFusion. I simply <cfquery>ied the database to pull all values from the original column, but CASTing the values as FLOATs. I then <cfloop>ed over that query and UPDATEd the table with the new values. SQL Server had no issues in going from FLOAT to NUMERIC(18,5). I was then able to drop the old column, rename the new column, and everything was good again.

I'm sure that if my SQL-Fu was better, I could have created a temp table, copied the data in there, done the CAST to FLOAT in the copy, then pulled from that temp table and updated the new column in the original table with the new values from the temp table. If anyone would like to provide me some samples to get started, or has alternate suggestions, I'm more than willing to give them a go.

I'm prepping to build out a consolidated registration site for a couple of the services that we provide at my place of work. Instead of replicating a registration process across a couple systems, we're building a centralized registration system and users can then take those credentials and log in to the services that they need. (We're not replacing existing user accounts with OpenID and OAuth or anything like that — sorry!)

There's a lot of work we'll be doing to make the process simpler and more elegant than the current set of processes. One major hurdle remains, however: how to prevent users from creating duplicate accounts.

When users register for a site — any site — and if they don't access the site right away or within the next couple days, they often forget their account information. If they wait long enough, they'll even forget the email address with which they registered. This is particularly problematic in online education, as someone may take a class, then not take another for a year or two, but return to take another class at some distant point in the future when their email (or physical) address may have changed. We don't want them to make new accounts because then we don't have a unified history of their activity (and this can cause problems with things like prerequisite courses).

A common approach I've seen and we've used in the past is to check against the provided email address. If there's already an account in the system with the given email address, then you let the user know that they already have an account and provide them a link to retrieve that information.

However, if the user is trying to register with a different email address, they're going to make a duplicate account. The system isn't going to know that bsimpson@hotmail.com and santoslhalper@gmail.com are the same person. So what to do?

I'm considering the following, but am very open to suggestions:

• Check the first and last name: if there's a match to the first and last name, suggest display a list of email addresses that match and say "Hey, if one of these is your email address, go ahead and retrieve your account info."
  
  
• Check the first and last name and city and state or country: It's very unlikely that there are two Janet Halzipools in Buffalo, New York, but one never knows. (In parts of South India, for example, a name like Omar Khan is very common, though less so in Reno, Nevada so maybe this approach won't work particularly well.)
  
  
• Check another unique identifier: We don't/won't/never will store Social Security Numbers, but if you have another unique identifier, you can always check against that. The University that I work for distributes unique identifiers to some, but not all, of our user base (hence the need for a separate set of authentication data). This would work pretty well, but if the user no longer has access to the email address which is associated in the system with this unique identifier, how do they retrieve their account information? Do you ask them to engage in the manual task of contacting support and then waiting for a response from support to get information updated in the system?

Once of my key considerations about any of the above strategies is this: if you find what you think is a match, how do you prompt the user to see if this is them without making them think their account security is at risk?

Again, any suggestions are welcome. Even if I can't remove duplicate account creation, if I can significantly reduce it with one or more of the above strategies, I'll be happy.

As part of the standard database normalization process, you want to store related items of data in a join table rather than a comma-separated list in a single row within a table. Instead of putting a list of a user's favorite colors ("red, orange, aubergine") in a single field in the database, you'd store each item in the list as a separate row in the join table. This results in easier querying and faster performance when trying to access this data -- except in one specific instance.

Oftentimes, you will need to run a query that returns a good deal of information about a user, including an item such as the user's favorite colors. If you store the user's favorite colors in a single field in the database, this is trivial, but it's not good form and eventually will result in all sorts of problems when you want to find all users who have, say, red as one of their favorite colors. So you store that information in a join table instead. However, when you want to get a complete set of data on a bunch of users at once, including the favorite color for each user, things start to get a little complex.

You can do a standard SELECT with a whole bunch of WHERE clauses to filter the data and do the join between the user information and the favorite color join table, but you'll end up with one result for every item in the favorite color join table. So if a user has three favorite colors in the join table, you get three rows in your select. How, then, do you make it so you can retrieve a single row for each user and have a comma-separated list of their favorite colors?

The answer, as I found it, came from the smart guys over at 4guysfromrolla.com. I'm not going to rehash the entire article, but instead point directly to the solution.

The solution is this: create a user-defined function in the database which uses the SQL COALESCE function to create your comma-delimited string. For example:

So this function takes a userID value and returns a comma-separated list of all the matches from the userColors table where the userID is a match. You can add additional WHERE statements in this function to filter the results further. The magic is the COALESCE statement, which takes all the results and combines them together into a single value.

This has to be set up as a user-defined function. You can't do this inline with other SQL in your main query.

Let's say your main query is "Give me all users in California and their favorite colors." You'd code the query like this:

The userID value in each row in the query gets passed off to the getFavoriteColorsList function that you created earlier, and it returns a comma-separated list of that user's favorite colors.

This is much faster than doing it in the application tier (ie; your ColdFusion or Java or PHP app). The drawbacks are twofold: it's not terribly generalizable — you'd have to create a user-defined function in your database for each such list you wanted — and you can't do a whole lot of fancy formatting as you might be able to in your application tier. Perhaps you could make the function generalizable (that is, pass in the column and table names as function arguments), but I don't believe that works. Please correct me if I'm wrong!

Sean Corfield covered the keynote at the Wee Dram of Scotch conference yesterday, where Adam Lehman talked about some of the possible new features in ColdFusion 9. To quote Sean:

The key point here is that Adobe is, potentially, looking at ColdFusion as the "hub" for a lot of development for a lot of different platforms. If core CF services (such as querying, mail, image manipulation and so on) are exposed as remote services, you can have PHP or .NET or J2EE applications calling these services. Interesting idea, though most other platforms have libraries which do many of these things, and do them natively. That may not be the case for everything (particularly AMF-based querying) and for every platform (Ruby and Perl don't, for example), but I think that developers who primarily code on these other platforms would say "Why use CF when I can grab a native PHP/.NET/J2EE library which does this for me?"

Exposing CF functionality as a series of AS3 libraries really does bind (pun intended) Flex to CF much more closely. That's definitely a good thing, as it increases the capabilities of Flex and keeps development strong within the Adobe family, while the more general Web services exposure of core CF services makes it available to other platforms.

And I'd personally love to see BlazeDS integrated deeply in to CF. It would make for faster, easier application building in CF and really help promote the use of the platform for data synchronization in rich Web applications. BlazeDS and LCDS is really nifty stuff, but it's a bit of a pain to configure, and there are some issues with CF object marshalling and management which need to be addressed. Deep integration of BlazeDS in to CF would knock down those two major barriers, and give the platform another needed bost.

It's good to learn something new every day. You can always do better. You can always learn more. Today I learned something that I suppose should have been obvious to me: given that ColdFusion is a weakly-typed, dynamic scripting language, you can pass functions as arguments to other functions.

This can be really useful for custom renderers: say you have a series of products but each need to be output in a slightly different way. You could write separate functions with a lot of repeated code to handle the rendering of each of the products, or you could be a good developer and pull out the repeated code in to a single function. You could be an even better developer and pass in a custom rendering function to the generic rendering function to handle the unique processing that each product needs for proper rendering. The generic function then calls the custom function that you passed in as an argument. This leads to simpler syntax, more flexibility, and reuse of functionality. Harel has an example of passing in a function as an argument on his blog.