A burst of several hundred comment spam appeared over the weekend. I have temporarily disabled comments, by the cunning means of adding the SCode plugin, which doesn’t actually work – not the fault of the plugin developers, but most likely a combination of my configuration of the plugin, and Apple’s bastardisation of blojsom.
We all have those days. The result of a long day’s labour is just one line of code. Well, that was my day for yesterday. And for today. I also wrote several debugging routines, and made quite a few testing changes, but they were all deleted from the final code check in, so they don’t count.
Mesa has had a couple of bugs in Excel import/export running on Intel. They didn’t show up in initial testing, but became apparent soon after. The obvious cause was byte ordering. Not so obvious was where.
However, Mesa has been working, and very thoroughly tested, on Intel since 1993. Also Sun, and HP PA-RISC, both a little later, of course. Longer than it has been running on PPC, in fact. But we rewrote the Excel support code in 2000/2001, when only PPC suppport was required; before that we only supported strings, numbers and formulas, but no formats. The new code took advantage of our existing byte ordering, but two dodgy lines of code crept in. One was my fault, in the re-port to Intel – I used a 16 bit integer routine where I should have used a 32 bit integer swapping routine, and that didn’t show up in the internal tests (because it worked for average, business-typical numbers).
There’s something like 20-30 hours of debugging in correcting these two issues, and that’s not counting reconstructing the project to match the new 10.4, Xcode 2.2 templates.
I have spent the last day updating Mesa. No new features, just reconstructing the project. The last time it was rebuilt was in 2000/2001, I think; required upgrades in the interim were done automatically whenever Xcode offered.
But a lot has degraded in the project since automatic upgrades started. I lost the ability to debug at some point after MacOS X 10.4; with my normal development style, this isn’t a great loss, but when moving to Intel support it was trying, to say the least.
It’s going to take some time to complete the rebuild. As well as getting back debugging, which I need to complete Excel import/export support, I’ve taken the opportunity to add in gcc 4.0 updates, and to take action to remove some of the additional warning that this shows up. I will also be updating to include new recommendations for placement of support files, etc – back when we made the decisions we did, we were following the same general practices as everyone else, and there were no specific guidelines from Apple.
As has already been well documented elsewhere, Zero link (ZERO_LINK) makes many projects break, Mesa included. The project inspector in Xcode ignores your Zero link settings; the only way I have found to reliably remove it is to edit the project.pbxproj file inside your xcodeproj wrapper; there seem to be two occurences for each target.
One thing that has made rebuilding more of a trial than it need be was localisation. For non-localiased files, you can drag and drop from project to project, from Finder, or use the Project->Add to Project menu option; but localised files have to be manually reconstructed. There is no reason that they couldn’t be dragged between projects, at least.
I will take advantage of the rebuild to generate a final build for old OS releases, probably targetting 10.2; after this point we will start to consider taking advantage of 10.4 unique features. So there will be a 10.2/10.3 release, PPC only, and a forward going version that will initially be 10.3 PPC/Intel, eventually moving to 10.4 only.
If I notice any thing else, I’ll be sure to point it out.
Tagged xcodeI have created two new articles in the static part of my site, one for general content management, and one specifically on adding RSS support to an existing WebObjects project.
There’s no rocket science; what matters the most is selection of database attributes for content; created/updated/published dates, creating/modifying user, and keywords. This is all exactly what we were doing 8 or so years ago in our first content management project, and it works. The blogging world like to call the keywords concept tags, and pretend that it’s new and dazzling.
The first article is relatively recently written, based on some code I wrote probably 5 years ago. It took me two days for the initial version, but I have invested quite a bit more in fiddling around with it since, all very unproductively – the initial version already had all it needed.
Adding RSS support in WebObjects is remarkably trivial, for normal applications (creating an RSS feed). It took me around an hour.
Tagged content management, rss, webobjectsApologies for various old articles popping up to the top. I’ve done a mass edit to convert blosxom-isms into plain html for blojsom, and it seems to have taken upon itself to do a shuffle. There are still a couple of glitches which I will want to correct, so this is going to happen again. The annoying factor is that, when I installed this weblog, I did some test edits to see if they would reorder, and they didn’t. The only obvious difference is that my initial edits were done using TextEdit (I think through WebDAV), and the latest large batch with TextMate.
So my next task was to move a forum from the old server to the new one. Download phpBB from http://www.phpbb.com and copy the expanded tar-ball to the web folder, then follow the standard installation instructions (which involve setting permissions appropriately for security and editing config.php). That part is easy enough.
Creating a backup from MySQL is also easy:
mysqldump -u user -p phpbb > phpbb.backup.sql
The -p flag prompts for the database password. Then use scp to move the backup file to the new server, and run
mysql -u user phpbb < phpbb.backup.sql
There's a catch. How to tell what the MySQL user is on a new installation. MySQL comes with 2 x 2 users - 'root' and '' (empty name), on localhost and your server name (`hostname`). You can change a password like this:
mysqladmin -u root newpasswd
This, however, only sets the localhost password.
Next step: create the database to restore to:
mysql -u root -p
create database phpbb;
quit;
Then run the database restore command given above.
phpBB2 has a few more installation steps - run the update script (in the install subdirectory, which one depends on what version you are upgrading from), delete the contrib and install subfolders, and check the settings in the Admin panel (link is /phpBB2/admin/).
You may also want to copy over any avatars in /phpBB2/images/avatars/ to the new site.
Tagged mysql, unix adminAny serious modern application should be email enabled – even if only for product feedback. It isn’t that long ago (ok, maybe it is) that I used to have a class to open a port connection and format its content into RFC 822 smtp; the hard part was picking up a server that would accept emails, but even that wasn’t hard – there were open sendmail servers back then (ones that were intentionally open, that is).
In the sequence of rewrites for Mail.app, Apple added in the Message framework with its NSMailDelivery class, which at first didn’t really work. But now it does. Then the mailto: URI format expanded from just opening a message window in conforming apps, to being able to handle a full mail message. Now that mail account information is considered to be a part of the user account (for MacOS X, at least), everything is available to make coding email sending as a simple part of any application.
I’m going to explain how to send an email in two different ways – first using mailto:, and then again, using NSMailDelivery. I’ll just touch on the basics, interested parties can research the options.
The mailto: URI takes a sequence of parameters; the first, and required, is the to address for your email. Like this: “mailto:email@hidden”. Other parameters need to be provided in a “key=value” format, joined to the URL in the standard convention, using “?” for the first keyword, and any remaining keywords joined using “&”. Like this: “mailto:email@hidden?SUBJECT=Hello&BODY=spamspamspam”. The entire URL should conform to URL encoding. This is described in RFC 2368, and can be used to construct any sort of email message.
In Cocoa, we need to create a URL with our full email message, and use this to create an NSURL object, then pass it to NSWorkspace to be opened; any app that supports mailto: can respond to this request.
NSString has methods to support URL encoding.
To the code:
NSString *encodedSubject = [NSString stringWithFormat:@"SUBJECT=%@", [[self subject] stringByAddingPercentEscapesUsingEncoding:NSASCIIStringEncoding]];
NSString *encodedBody = [NSString stringWithFormat:@"BODY=%@", [[self bodyString] stringByAddingPercentEscapesUsingEncoding:NSASCIIStringEncoding]];
NSString *encodedTo = [[self to] stringByAddingPercentEscapesUsingEncoding:NSASCIIStringEncoding];
NSString *encodedURLString = [NSString stringWithFormat:@"mailto:%@?%@&%@", encodedTo, encodedSubject, encodedBody];
NSURL *mailtoURL = [NSURL URLWithString:encodedURLString];
[[NSWorkspace sharedWorkspace] openURL:mailtoURL];
This is an extract from a utility class with instance variables that encapsulates the task of sending email from a standard application. You can drive this programmatically, or by binding into your user interface.
The next method is to use NSMailDelivery. First, include /System/Library/Frameworks/Message.framework in your project (make sure that it is set to be an absolute path), and then open the framework and read the NSMailDelivery header file.
The deliverMessage class method is all we need for normal emails. It takes an NSAttributedString for the body content (so it supports rich text), and a dictionary for headers. If you want to add further headers, perhaps cc, bcc, or even your own customer headers, just add them to the dictionary, and everything will be added as a header. It is perfectly possible to create fully rich content emails this way, with attachments and so on. The format parameter supports both MIME and ASCII format emails. Example:
NSMutableDictionary *headersDict = [NSMutableDictionary dictionary];
NSString *theMessage = [[[NSMutableAttributedString alloc] initWithPath:@"/tmp/ExampleDocument.rtfd" documentAttributes:NULL] autorelease];
[headersDict setObject:[self to] forKey:@"To"];
[headersDict setObject:[self ccArray] forKey:@"Cc"];
[headersDict setObject:[self subject] forKey:@"Subject"];
BOOL result = [NSMailDelivery deliverMessage:[self body] headers:headersDict format:NSMIMEMailFormat protocol:nil];
Note that the default from address (in both cases) will be taken from the default mail account for the user. The NSMailDelivery api also allows you to set the mail account to be used as sender.
You can download an example application including all the code and the utility class mentioned above.
Tagged cocoaAs I write this, I am (still) in the middle of implementing the MacOS X Server blogging tools. I have used everything from TextEdit (formerly Edit.app) to blosxom and even LiveJournal and my own custom content management systems to manage content, so I am understandably nervous about using a system that, judicious reading of the blojsom and Apple’s Collaborative Services mailing lists will reveal, is confusingly crippled. However, I have reasonable confidence that a system based on blosxom could be expanded to meet my general needs, and that blosxom has atrophied enough that it no longer reasonably can do so.
So, what are the problems? You need to edit config files (all stored in /Library/Tomcat/blojsom_root/webapps/ROOT/WEB-INF) to delete blogs, and to enable the admin ‘flavor’, which lets you modify a few things that Apple doesn’t. The biggest problem is that you get four default themes, and they are all very Appleish. They also silently “lose” any posts beyond the page limit – they are still there, but you can’t follow the category summary back by page.
It doesn’t seem possible to install the standard blojsom ‘themes’ as alternatives; this may be because my knowledge is limited; Apple’s version appears to be limited to changing the css stylesheet. The Apple version doesn’t include the permissions.properties file, which effectively makes it impossible to configure plugins.
Some people recommend installing the latest release of blojsom alongside the Apple version, and using that – upgrading is not a feasible option, apparently.
To enable the admin flavor – for each user folder in the main config folder, edit flavor.properties to uncomment the line beginning ‘admin=’. The standard blojsom admin console is extremely hideous, with an ungainly UI.
Deleting a blog: edit blojsom.properties, the line starting ‘blojsom-users=’.
Apple seem to have made the permissions.properties file vanish; which makes it impossible to enable extra plugins. Now, they may have moved it into some part of the Open Directory authentication – which I am investigating, but can’t see any traces so far.
Using the standard blojsom themes may not be so easy. So far, I have a couple of test blogs set up; these will disappear as I work out how to make new styles/themes work.
Here’s a little trick: there are several ways that you can make a blog the main page for a site. The best overall solution is to use Apache rewrite, but the second best is this. Create an index.html that uses your blog as a single embedded frame, like this:
<frameset cols="100%,*" border="0">
<frame src="http://www.plsys.co.uk/weblog/meca/news/" name="Meca">
<noframes>
<a href="http://www.plsys.co.uk/weblog/meca/news/" name="Meca">Meca Web Site</a>
</noframes>
</frameset>
You will only see the master URL in the browser window.
Another trick: uploading files. First, enable the admin console. Access it by appending ?flavor=admin to your blog’s URL. Look for Weblog Settings -> Files, and upload your file. You can access it with something like:
<img src="/blojsom_resources/meta/blogname/image.jpg">
The File Upload admin plugin is supposed to show you a link that you can copy to see the file, but I don’t see it.
To customise a ‘theme’, as Apple calls it. You can install the standard blojsom themes by flattening out the /Library/Tomcat/webapps/blojsom/WEB-INF/themes/themename/ folder from the current blojsom distribution, and putting the vm files into the /Library/Tomcat/blojsom_root/webapps/ROOT/WEB-INF/templates/ directory, and the resources into /Library/Tomcat/blojsom_root/webapps/ROOT/blojsom_resources/meta/blogname/. You must edit the flavor.properties file in the blogs folder to set the html= line to your installed theme. I had to do some editing, as the BLOJSOM-SITE-URL was messed up in the default Apple install (or in the vm files, take your pick). On the whole, it is probably simpler to stick to revising the css stylesheet, which can be handled by editing the properties files.
One trick: I wanted to use see-me-more, which concept I am used to from blosxom. But if I added it onto the end of the html chain in plugin.properties, it ignores my preferred tag. This shows up in the post text, because it comes after the escape-tags filter. Just more the filter to before escape-tags, and all is happy again.
Tagged mac mini, unix adminMy current server runs FreeBSD, sendmail, and cyrus IMAP. I experimented with an old (more than 5 years) webmail package, but that hasn’t been in use for some time. It has a procmail mailer of my own devising to handle some primitive spam filtering, but that really doesn’t cut it any more.
Moving to MacOS X Server means a change to postfix, working webmail (SquirrelMail), filtering (spamassassin, clamAV, and Sieve), and using MailMan for all lists. The change from procmail to Sieve doesn’t bother me, and the webmail won’t make a significant difference. We had MailMan installed, but dropped the couple of lists we had that used it several years ago.
A couple of things are giving me a headache: the first is users. I have set up my user accounts and given them my own randomly generated passwords, which I can tell the users. However, I really want them to be able to change their passwords themselves, and I don’t see a way of doing that beyond giving them shell access, which isn’t going to be appropriate for all users.
The second is aliases. We use a lot of aliases: paul is mapped from Paul_Lynch, paul.lynch, and a few others; we also use aliases for common list type addresses: info, sales, etc. I can see using MailMan for the lists, but it isn’t appropriate for single member lists. In sendmail, I just edit the aliases file and re-make /etc/mail: end of story. But I don’t see a UI tool to do the same for MacOS X Server. Apparently you add additional short names in Workgroup Manager to handle this; the first short name is ‘special’. This doesn’t help for account redirections, including redirecting from an alias to a list.
The normal solution for postfix is to edit the /etc/aliases file and run newaliases, which is very similar to the old sendmail solution. So MacOS X Server supports single address aliases through the UI (what may be called ‘real’ aliases), but traditional aliases still have to be created the old fashioned way – call them ‘group’ aliases.
In addition, postfix supports most existing sendmail configuration files; we also use virtusertable for off-site (ie non-user) aliases and groups. The aliases file is restricted to local users only.
As we have a number of domains, I will need to investigate the ‘advanced’ features for virtual hosting, could be interesting.
Notes:
If you have an alias similar to “Paul_Lynch”, it must be entered into the short names as “paul_lynch”. This is different from the old netinfo form of mail aliases, and definitely caught me out.
For spam filtering, you need to prime the pumps by creating two mailboxes, one for spam and one for definite non-spam, and running a command line on them. Once that is done, you should create users called junkmail and notjunkmail; use forward (Cmd-Shift-E) to send new spam to these account, which are processed by cron every night. After about three weeks of this, I still have a lot of spam coming in, but it does seem to improve gradually.
More detail on aliases
/etc/aliases is linked to /etc/postfix aliases. When you make a change, you must run newaliases, and then ‘postfix reload’.
Postscript:
I have been running, reasonably painlessly, on this server since January 1, 2006. The transition went more smoothly than I expected; the only surprise was the one mentioned above about alias case sensitivity.
Somewhere I kept some more notes about exact configuration steps, which I will post when they surface.
Tagged mac mini, mail, unix adminMacOS X has had some strange changes made to it recently. Take vacation, for instance. I believe that the version used by MacOS X came from the sendmail distribution, which is perhaps the reason that it has vanished from recent versions of MacOS X. The net effect is that creating a simple autoreply vacation message is next to impossible.
If you check the MacOS X Server documentation, you’ll find that Sieve is installed, and you’ll even find an example script for vacation, which sounds great. But then the Sieve installation on MacOS X isn’t usable – you need at least to install timsieved and enable the sieve tcp port to provide reasonable access to create sieve scripts. What is more, having user installable scripts for MacOS X users as the only way of maintaining an autoresponder isn’t acceptable.
There are plugins for SquirrelMail that support the vacation program, which is great. But they still need vacation installed; the combination of both is exactly the solution needed.
First, installing vacation. I found sources for a maintained version of vacation on a Debian site (see below), but there are several sources available; I simply chose to use the first one I found.
To install, I removed the LIB option from the Makefile, and editing the vacation.c file to change the location of sendmail from /usr/lib/sendmail to /usr/sbin/sendmail. This resolves to the link that postfix puts there to handle mailers that try to use sendmail directly. Then just run make, and manually install vacation to /usr/bin/vacation; you can use other locations, but that one is traditional, and you would have to edit vacation.c further to add in the new path.
That seems to work.
For SquirrelMail, you need to first enable WebMail, which is done in Server Admin in the Web section (not the Mail section, which had me confused for a while). You can then access your mail via http://servername/WebMail/, and just use your normal mail login details (same as account login for simple configurations).
Next step is to install a vacation plugin from squirrelmail.org; I chose vacation_local, as it seemed to support MacOS X Server configurations, and had the most downloads. Unpack the tarball into /etc/squirrelmail/plugins, and copy config.php.sample to config.php; I had to edit the ftp server location from localhost to the address of the server – if I didn’t do that, entering the vacation options panel in WebMail generates a PHP ftp error. I am guessing that it defaulted to IPV6, and that somehow isn’t supported – but that’s just a guess.
You can now set up your vacation details by taking the Options menu choice in WebMail, and selecting Vacation.
You should now have a working autoresponder, and have configured WebMail as a byproduct.
Tagged mac mini, mail, unix admin