• ZOHO Creator – 3 apps free. $15/mo basic plan.
• Dabble DB – free public data. $24/mo for private data. Very cool concepts.
• Web Office – $53/mo includes data.
• Google Docs – Free. No database. Spreadsheet only.
• BaseCamp – No free option. Starts at $24/mo.
• Windows Azure – Clearly not personal. Impossible to tell what it would cost.

And there are more, mostly pitched at small businesses and enterprises, and expensive. At least very expensive compared with Blogging or Photo Galleries. So my question is: Why are database tools so expensive compared with Blogging and Photo tools, which are usually free or almost free? The reason I believe is Open Source.

I think there is a very interesting opportunity for a platform, like WordPress for personal data tables.

Like WordPress for Data

WordPress is a very popular blogging platform and it has an Open Source model. The software is free, but there is a combined Hosting Service / Blog based on wordpress that costs a modest amount. It seems to me that we need to have this model with web data tools, or what I’m going to call Personal Cloud Databases. What I want is a WordPress like platform for Personal Data. It would have the following attributes:

1. The software would be OpenSource under GPL licensing and
2. would be suitable for personal and small business use. The target is not the Enterprise, those folks already have many cloud solutions.
3. You could easily install it on your own hosting service or
4. there would be one or more paid services that provide reasonable plans – $5-$10/mo – that would provide hosting possibly with co-located blog/sites and domain purchase.
5. Hosting sites like SquareSpace could integrate the software to provide data services in addition to their site creation and hosting.
6. The platform would allow import and export of data in some common forms, including SQL so that more complex databases could be accomodated.
7. The methodology would be simplified. More at the level of MS Access or OpenOffice Base, rather than the level of SQL tools or phpMyAdmin.
8. WordPress features of Themes, Plugins, Widgets would be provided to allow a rich eco-system of contributors to develop. The real success of WordPress, I think, is the nature of this large eco-system. And themes, and plugins are key to this eco-system.
9. WordPress features like ease of install and ease of update are provided. The ability install plugins/themes directly with no download/upload cycle and the ability to upgrade the wordpress directly on the site are key ease-of-use features for any modern web platform.
10. It would fit nicely with, or embed in, Joomla, Drupal, WordPress and Forums. This means that it can be embedded to add data, as well as view it, and it can have complementary themes, or use, themes from these CMS systems – and possibly others. Joomla and Drupal do not have serious data management plug-ins, and the use of these CMS systems to build serious business sites points to the need for a data tool that works with these environments.

Starting a Design but DHX Quick Tables is Better

I have started design of such an environment as I just described, based on the DOJO Javascript toolkit and a php back end. But then I found the DHX tools and specifically DHX Quick Tables and I’ve re-directed my efforts.

At this point, I’ll be forwarding these ideas to the creators of DHX tools and continue, at some level, to investigate using DHX tools to build a Personal Cloud Database.

DHX QT Enhancements for Personal Cloud Data

As I understand it, DHX QT has the following limitations that should be enhanced to provide the PCD platform:

1. Ability to build tables and fields directly in the tool rather than importing tables that are created with another tool.
2. Support for field types. See below.
3. Support for forms. While one can to CRUD [Create Update Delete] on a table using only a grid view, there are more complex tables [or support for long text fields like the book example] where forms are useful.
4. The elimination of a table creation step in favor of table creation as a form is created has much appeal in simplifying the process. ZOHO creator does this. Study of ZOHO Creator may lead to interesting insights into the entire design.
5. Support for multiple users and passwords in the style of WordPress.
6. Support for multiple databases under one installation. Currently DHX QT supports only one database per installation, so a separate installation is required to support multiple databases.
7. Storage of meta data in a database table rather than a text file. This makes backup easy. Currently backing up a DHX QT application requires backing some some configuration data files.
8. Themes. Incorporating the Skin Designer into the platform and allowing Themes to be exported would solve the problem. Of course apply and manage themes
9. Ability to easily embed table views in other CMSs / web pages. Export an <IFRAME> at minimum.
10. Plug-ins, Auto-install, update etc. as for WordPress.
11. Plugins probably take the form of filters and datatype widgets. Complex layout like wordpress is not needed since the data tables can be embedded in other CMSs to provide this layout.

Support for the following datatypes is interesting. These are pretty much those provided by ZOHO Creator:

1. Text with max width.
2. Multi-line text.
3. Text with an input mask.
4. Numeric field: Integer, Decimal, Real?
5. Date, Time and Date/Time. All with popup widgets for setting and display preferences.
6. Enum – with an easily edited set of choices. Display as Pull-Down or Radio box as edit.
7. Set – with an easily edited set of choices. Displayed as a set of checkboxes when modifying the field, rather than requiring “armor,pants,gloves” style entry.
8. Single Checkbox.
9. Image – as file path on the server or url or blob.
10. URL – as link to web with New Window option default.
11. Table Link or Lookup field. This relates a field on one table to a field in another table. This provides the ability to build 1st normal form databases. For example, A Char_Table contains an ID and a CharName field. In another CharArmor table, the Char holding the armor is specified as a CharID field which contains the ID number from the Char_Table. In the UI it is displayed as the CharName text field and is set with a drop-down populated by the CharNames from the Char_Table. ZOHO Creator has a Lookup field, which appears to work this way.

Most of the complicated parts of the platform are already present in the DHX tools, including the widgets – maybe not image – and the grid display and “connector” which provides a simple way to build a server in php, or other languages.

I would hope that the DHX folks pick up the torch and proceed to build the environment as an Open Source platform in the style of WordPress so that we can get on with building the eco-system.

Enjoy,

Windy

I have found a solution with DHX Quick Tables, and for now I’m using that as we will see. I’ve exported the 2Moons data from Zoho Creator and built a MySQL database with 3 tables: character, armor and weapons. I’ll do the other tables later, but for now everything is working. I’m using MySQL 5 and innoDB tables. One minor glitch is that DHX requires php5, and I had not yet upgraded my site to php5. Once I asked the support group how to upgrade, I was all set.

I’ve discovered the following things that will help you if you decide to do this as well:

First you need to define  your MySQL database and table. When testing this locally in an xampp environment, I used MySQL Admin to define the database, and a user with a password and full access to the database. On my hosting service, I used phpMyAdmin, which is provided in the control panels. Once I had the database and user defined, I used phpMyAdmin to create the tables and upload the data. I had exported the data from ZOHO creator as CSV files, so I stripped off the header and used the LOAD function to populate the table. There are several things to watch for as you migrate to DHX quick tables.

1. You need an INT autoincrement field in the table. This field will not be displayed in the tables, but is required so that updates work correctly. It is easy to add a field using phpMyAdmin. Set the datatype to INT and enable AutoIncrement and Primary Index.
2. Avoid integer fields, unless you want to enter data. Null fields for INT fields do not appear to work. But VARCHAR NULL fields work just fine.
3. SETs and ENUMs work, but you cannot leave ENUMs null either, so you may want to use SETs. ZOHO provides a series of checkboxes for ENUMs which is more convenient to edit them. With DHXQT, you must enter them as a series of names with commas. Leave out the spaces like this: pants,armor,helmet.
4. You can set a character name using a pull down list.
5. Linked fields are not provided. The tables are isolated. This may be a problem for some applications, but in this case it’s not a big deal.

Table Definitions

Unlike ZOHO creator, MS Access, or OpenOffice Base, defining tables in DHXQT requires knowledge of SQL and detailed knowledge of databases. As I will discuss in a later article, DHXQT could form the basis of an easy to use Personal Cloud Database tool, but it’s not there yet. You must be database savvy and comfortable with tools like phpMyAdmin to use DHXQT.

For this application I’ve defined three tables:

1. A character table.
2. A table with Armor parts.
3. A table with Weapons.

Here’s the Characters table:

--
-- Table structure for table `characters`
--
CREATE TABLE IF NOT EXISTS `characters` (
`ID` int(11) NOT NULL auto_increment,
`char_name` varchar(32) NOT NULL,
`level` int(11) NOT NULL,
`class` enum('Aloken','Knight','Summoner','Segnale','Hunter','Magician','Bagi Warrior') NOT NULL,
`server` varchar(16) NOT NULL,
`account_name` varchar(32) NOT NULL,
PRIMARY KEY  (`ID`)
) ENGINE=InnoDB  DEFAULT CHARSET=latin1 AUTO_INCREMENT=12 ;

The armor and weapon tables are more complicated:

–
– Table structure for table `character_armor`
–

CREATE TABLE IF NOT EXISTS `character_armor` (
`ID` int(11) NOT NULL auto_increment,
`char_name` varchar(32) NOT NULL,
`level` int(11) NOT NULL,
`armor_type` varchar(64) NOT NULL,
`armor_parts` set(‘boots’,'pants’,'gloves’,'armor’,'helmet’) default NULL,
`boots_fortified` varchar(64) default NULL,
`pants_fortified` varchar(64) default NULL,
`gloves_fortified` varchar(64) default NULL,
`armor_fortified` varchar(64) default NULL,
`helmet_fortified` varchar(64) default NULL,
`sold` set(‘boots’,'pants’,'gloves’,'armor’,'helmet’) default NULL,
PRIMARY KEY  (`ID`)
) ENGINE=InnoDB  DEFAULT CHARSET=latin1 AUTO_INCREMENT=80 ;

–
– Table structure for table `character_weapons`
–

CREATE TABLE IF NOT EXISTS `character_weapons` (
`ID` int(11) NOT NULL auto_increment,
`char_name` varchar(64) NOT NULL,
`level` int(11) NOT NULL,
`weapon_name` varchar(64) NOT NULL,
`have_need` set(‘have’,'need’) default NULL,
`plus_level` varchar(4) default NULL,
`enhancements` varchar(64) default NULL,
`gem_fortifications` varchar(64) default NULL,
`sale_state` set(‘sale’,'sold’) default NULL,
`dil_price` varchar(11) default NULL,
PRIMARY KEY  (`ID`)
) ENGINE=InnoDB  DEFAULT CHARSET=latin1 AUTO_INCREMENT=115

The quick tables created from these look as follows:

The Character Table:

The Armor Table:

The Weapon Table:

As you see, each instance of DHXQT supports multiple tables from the same database.

You can have as many databases as you like on your system. Just install a new instance of Quick Tables for each database.

Applying New Styles

As you can see, I have set the title tag and adjusted the styles to be the built-in style of black. This must be done by editing the php file for each table manager.

The DHX Skin Builder is used to build the styles and then they are copied to the DHXQT directory and the table file modified to load the new styles.

The Books Database

Two years ago I needed to sell a couple of hundred old science fiction paperback books, so I entered them all into a database table, photographed their covers and divided them into lots by publication years to sell on Ebay. At the time I wanted a way for folks to see the detailed information on the books before they bid. I built a MySQL data table by hand and then a simple php application, again by hand, to sort and display the books in a web page. The web display is trivial using DHXQT, but there are limitations. It does not seem there is an image display widget in the DHX toolkit this is a surprising omission given the power and breadth of the toolkit.

Here is the book table in QT.

Summary

While Quick Tables does not have linked fields, and some of the other  field types that ZOHO supports, Quick Tables have the advantage of being free, so you can build as many tables as you like. They are slightly more difficult to build, but there is no limit to the number or the size of the data you can store. Given the key scheme that Quick Tables uses for security, the data is quite secure from modification.

- windy

In some previous cases I’ve needed simple databases: Books for sale on EBay, a DVD database list and a 2Moons Character database. For Books, I used MySQL and php here on this site [ But I didn't maintain it.] For the DVD database, I used Open Office connected to MySQL to store the data and image thumbs of covers. For the 2Moons characters, I used an Open Office Base application.

When I selected the Open Office solution to avoid high costs and to allow me to move the application around between computers and operating systems. Open Office works on Windows, Mac and Linux. However, the problems of backing up and saving versions of the application as I enhance it are a lot of work. Which version on the various computers is the latest one? Sure would be nice for it to be in the Cloud so I wouldn’t have to worry about those issues.

After a little searching, I found ZOHO Creator. Here’s two views of the 2Moons database that I build in an hour or so.

So it seems to work pretty well. Here is a list of weapons filterable by Character name.





CAUTION: Do not use the JavaScript snippet. It breaks WordPress layout.

One of the issues that I struggle with in OOBase was building a form that filtered by character. I had to get an obscure script from a fellow in a forum to solve the problem. But with ZOHO creator, it was easy. Just drag a LOOKUP field into your form and set the field. Problem solved.

You’ll notice that he embedded views look pretty generic and don’t match this website at all in style. Sure would be nice if ZOHO provided “Themes” or worked with the stylesheet of WordPress or other CMS where the embedding occurred.

I did find a minor problem. If you build a Dropdown list field, and then later you try to add or modify the choices, Creator will likely kick out some errors. Based on a forum post, they will consider this as a future enhancement, but as a workaround, you can edit the form script to fix the order of the entries.

ZOHO provides useful and free subscriptions for each of their Office features. For Creator there is a free subscription that allows 3 applications, 1000 records and 200MB. But if you want more, the next upgrade is $15 / month. Sigh… There is a reason that I don’t spend $15 / mo to play World of Warcraft. That seems way too expensive compared with my hosting service which gives me UNLIMITED MySQL databases and UNLIMITED storage for $5 / month.

But the good news is that it works just fine and is easy to use, even for a novice with limited database experience. Force.com provides a solution too, but their solution requires a database administrator level of knowledge to use. ZOHO is much better.

You can share your application for $$ or for free, with or without the data, and with or without the ability for them to modify the application. Pretty slick. They also have a place for developers and clients to meet base on requirements of work that clients need done.

- windy

Managing a stable of characters in an RPG is a complicated process. My limited experience with MMORPGs indicates that it can be more efficient to build more than one character at a time. At least it is more interesting. In the two MMORPGs that I’ve tried I’ve built several characters to try to learn more about the game and the character classes. In 2Moons I’m currently running six characters. This is efficient since as they level up all the interesting loot is saved for the other characters and one higher level character can finance a lower level character’s purchase of expensive items.

But, the downside of doing this is you have to keep track of all the loot that each character has and what they need. In Guild Wars this was easy because there is an account-wide storage facility where all the loot is kept. But with 2Moons, each character has a separate stash, or storage locker, so you have to remember what’s in each one to avoid wasting time with items that you already have collected.

I’ve started to build an Open Office database to store this data for my 2Moons characters. I’ve got the data scheme designed and some reports done, but at this point I need Forms to fill out the various data. The data is in Normal Form and I’m not sure how to build a form to allow entry of the data. First let’s review the data design and the queries.

This is the character table.

This is the armor table.

The details are not important unless you are a 2Moons player. There are too many types of armor to have a table for that. Instead of a Armor Name Table, we just list the armor name. Each armor type has 5 matching pieces. Some intermediate armor types only have one or two parts, but we ignore that detail. This table allows us to keep the status of each bit of armor that we need and which pieces we have.

But the interesting case is the stones case. For the collection of stones, there are only a few types so that there is a table of stone names. And we want a report of the number of stones kept by each character.

This is in fully normal form, just a list of the character id, stone type id, and the number of stones.

Here is a query that gets the list for a report:

So that is clear enough.

But what about the forms to fill out the data. So far I’ve been filling out the data manually by editing the tables. This is ok for the characters, but for the Armor, Weapons and Stones, this is not convienent since the numbers of the character records and stone types have to be entered manually.

I’d like to have a form for the Armor that allows the Character to be set with a pull-down, and a Stones form that allows the Character and Stone Type to be set from pull-down lists.

Can someone show me an example of this?

Here is the Open Office Database file: 2moons_characters_example

Thanks,

- Windy

Update:

Thanks to VooBase on the OpenOffice.org Forums for suggestions and pointers to using a listbox to solve the problem. I have solved the problem and the forms work just fine.

Here are the forms to modify the Armor, Weapons and Stones held by each of the characters. The characters entries can be made or modified by using any of these forms. The list of stone types has to be modified manually by editing the table at this point, but adding a form to do this is trivial.

[All the data in the database is correct- these are my 2Moons characters. But the account names have been modified obviously, for security reasons. See you in game if you like. ]

Here is the Armor Form:

By selecting any field in the character field above, you can scan through the characters and modify any fields you like. By selecting the subform below, you can add or modify entries in the armor table. Note that the character field is a list, so you can pull-down from the available characters. This allows you to move armor among characters. Note that the armor is sorted by the highest level first, which is normally the most interesting armor.

Here is the Weapons form with the character list pulled down:

This form works the same as the Armor form.

Here is the Stones Form. This form has two pulldown lists, one for character and one for stones type:

And here is the Stone_ID column open showing the settings:

The Wizard does not work in the Spreadsheet format of the subform. But I was able to create a field in the form using the wizard and then copied the settings from the List Control to the column to get it to work. The character_ID column control settings are identical to the separate control. I wrote the Select statement for the Stone_ID column by example from the Character_ID field.  I have no idea what the bound field means.

The other issue I ran into is how to add the label for the entire form above the fields. To solve this problem, I changed the anchor point for all the controls to Page and then moved them down. Then created a label, anchored it to the page, moved it and set the font to size 24.

Each time you save the form, you need to go to the database dialog and save it all too. If something crashes without you doing this, you will lose all your work, as I did. BTW, this was built with Open Office 2.4.

Here is the updated example:2moons_characters_example2

Enjoy, if this application helps you with 2Moons or in any other way, I’d prefer an acknowledgment and a note saying how it helped. DISCLAIMER: Like all software you find, your use of this is at your own risk. If there are bugs, I’ll help if I can.

Cheers, and see you online.

- windy

When building a database of my DVD collection which includes BLOB images of png images of the covers, the database can be backed up, but it cannot be restored. This of course makes the backups useless. The images are 475×475 pixel PNG images.

UPDATE: Problem solved. Parameter max_allowed_packet is the packet size for requests. This defaults to 1M and needs to be larger to allow this restore to work. Setting the max_allowed_packet parameter to 10M allowed the restore to complete successfully.

I’m using MySQL administrator 1.2.12, MySQL 5.0.51b-Community-nt, MySQL client version 5.1.11 running on Windows Vista Ultimate x86 SP1.

The database was built using OpenOffice 2.4.

Using the default backup settings, the error on restore was:

Error while executing this query:INSERT INTO `dvd-collection` (`dvd_ID`,`MovieTitle`,`Director`,`Producer`,`Actor`,`Genre`,
`Notes`,`Rating`,`Review`,`Subject`,`Writer`,`Format`,
`Source`,`DateAcquired`,`ReleaseYear`,`MPAARating`,
`Length`,`disk_image`) VALUES
(1,’Step Into Liquid.’,'Dana Brown’,NULL,’Robert August,
Rochelle Ballard, Shawn Barron, Dave Kalama, Gerry Lopez,
Robert \”Wingnut\” Weaver’,'Documentary’,
‘No stuntmen. No stereotypes. No other feeling comes close’,
5,NULL,’Surfing’,'Dana Brown’,'DVD’,'DishNetwork’,2006,2003,
‘PG’,88,0x89504E…44AE426082);
The server has returned this error message:MySQL server has gone away
MySQL Error.

Using these additional settings:

• InnoDB Online Backup
• No EXTENDED INSERTS
• Add DROP Statements
• Complete INSERTS
• Comment
• Compatibility Mode
• Disable Keys

The error was:

Unknown object in backup file

Download excerpts of the SQL files that fail to restore:

mysql_restore_bug 1.4MB

DVD_Backup_2_40_Rows 14MB

These excerpts fail as well. The complete SQL backup file is 270MB, so not only is it large but I don’t really want to share the entire file. By the way, “Analyze Backup File Content” works fine for all the files.

The mysql command fails as well:
C:\Darrell\Video_Database\backups>mysql --user=root --password=mysecret dvd_video < DVD_Backup_2_40_Rows.sql
ERROR 2006 (HY000) at line 89: MySQL server has gone away
C:\Darrell\Video_Database\backups>mysql --user=root --password=mysecret dvd_video < DVD_Video_Two_Rows_Second_Try.sql
C:\Darrell\Video_Database\backups>

It works for 2 rows, but not for the 40 row file.

Thanks,

Windy