Upgrade my Mac OS X Server to Lion
Attention: this blog entry is in reverse order meaning the task started at the bottom and I’ll add updates on top of it
Fri Aug 12: Today wa the day where I pulled the plug from our Snow Leopard server taking it from the Internet. I did that to make sure that any new mails sent will not be received until the migration is done otherwise I might loose some of the emails. Then I also wanted to make sure that I don’t loose any changes to the web sites like comments etc.
So after the Server was taken from the Internet I exported the Mail DBs, the MySQl (WordPress) and PostgreSQL (XWiki) DBs and copied them onto another drive. The last step of the preparation was to make a SuperDuper copy and test the copy by booting from it. This way I was sure that I could recover the server in case of big problems and or if I needed to export more data. After I rebooted into the SuperDuper copy I also used it to reformat the original server disk by earsing it with the Disk Utility and then start the Lion installation. It turned out that I still needed an Internet connection but because I have a strange double router setup for the server I was able to connect the server to the Verizon router which prevented the Mac Mini to be identified as server. So I could install Lion and after that was finished the Lion Server and Lion Server Administration tools. After I installed some necessary programs like Dropbox, 1Password, pgAdmin3, MySQL and MySQL Workbench as some others and added the localhost as interface to the embedded PostgreSQL I restarted the box the first time.
The first problems I encountered was some issues with the SSL certificates but eventually I could generated a self-signed certificate and import it to the server. Then I ran into issues with Mail and DNS. Eventually I had to setup DNS step by step. First the local server definition, then the machine, than the aliases etc. And between each step using nslookup to verify everything fine. After all was setup I went ahead and setup the Mail server. Copy over the original Mail DB was that difficult after I figured out that I had to take about all the “.” directories by copying with “cp .*” because a “cp *” would not find and copy the “.” directories. Then I also had to make sure that all files had the ownership “_dovecot:mail” otherwise mail would ignore them. Eventually I got Mail up and running and the old mailboxes copied over. Still there is one thing that doesn’t work. I cannot use SSL with the Mail server and so I can only use Mail inside our home to prevent people from snooping on our mails.
Copying, installing and configuring the web sites was a breeze including the import of the MySQL and PostgreSQL DBs. This was done in less than 2 hours.
There are few things to do like installing Subversion and Subersion server, Gitolite server, Time Machine drive and some file shares but that is purely internal stuff and can be done when needed. Good thing there is that I have a life copy of the server on the SuperDuper backup and so I can check how it was done then and copy necessary files over.
I guess that concludes this post except I ran into some important issues or when I could figure out why Mail doesn’t support SSL for now
.
Tues Aug 9: Yesterday I manage to migrate over my Snow Leopard Mail DBes. It seems that the data structure is compatible and the only thing that I had to adjust is the user (moved to ‘_dovecot’). The rest was just sending one email to myself to create the user DB, then shutdown the mail service and copy over the data. After a restart the mail was available. There was one little thing where sub mail boxes where not handled properly but a many copy of these directories did fix the issue.
Today then I was able to install MySQL (just use 5.5.1 for Snow Leopard) and export / import the PostgreSQL and MySQL databases without any problems. Finally the website were easy as pie. Just tar up the directories in question (Jetty and wordpress), install on Lion Server and restart.
Next step is to actually do the migration. So first take it from the Net (not to loose any mails etc), then backup with SuperDuper, export the Mail DBs and regular Databases and export them onto an external drive. After reformatting the server’s drive I can install Lion, Lion Server, Lion Server Admin tools and the additional programs like Dropbox, 1Password, PathFinder, SuperDuper, MySQL, MySQL Workbench, PgAdmin3 and Java (triggering with ‘java -version’ on command line). After that I need to setup DNS, Mail and Web Sites, import the data and test it. I expect that DNS / Mail to give me some grief (as it always did) but on a late evening / nite shift that should be done. So see you on there other side.
Sun Aug 7: I came up with a plan to test the migration and to make sure that I can export / import the DB and Mail data before doing the actual upgrade. So I am taking my laptop, install OS X Lion Server on it and start the migration process. This includes the setup of the server so that it works the same way as my current Snow Leopard server. The only thing that I am going to drop is to use managed clients feature because that wasn’t worth the time I invested into.
I did figure out how to use the embedded PostgreSQL server of the Lion Server and it turned out that I cannot use the installation from Enterprise DB because it doesn’t work and the uninstallation does corrupt the internal PostgreSQL DB. Not good.
This will be a post in progress because the upgrade of the server will take a week or two. As soon as I learned how the OS X Lion Server was distributed I bought the Server App from the App Store. Compared to the original $1,000 (for Leopard) and $500 (for Snow Leopard) this time the server was cheap around $50. So I was ready to test it on one of my developer partion.
The first thing that I saw when I fired up the Sever is that the list of services where limited:
For me the most important thing was the missing DNS and Open Directory service and the limited functionality for the Web and Mail. So I went on the Internet to see if I missed some settings or flags but it turned out that one can download the good old OS X Server Admin tools which provides the missing services and the Mail configuration. Still need to figure out how to setup Apache sites I did on Snow Leopard Server. Because I don’t want the server to be down for too long I probably go ahead and use my Laptop and an external drive to create a server clone. When this one is up and running as expected I know what to do. Then I will shutdown the server, install Lion OS and the Server, copy over the necessary data (PostgreSQl DBs, Mail, Web Applications etc) and finally bring the server back to life.
– Andy
Strange Debugger Issue with Local Variables
Update: tested against the latest XCode 4.2 build and it is still the case. So I reported a bug:
Update: after I wrote and published this entry I realized that what I suspected might be right. So I made a breakpoint inside the constructor of the class in question: ProgramStepEntity and later when it is used as local variable. Now all the variable show up in the constructor (actual init method) as well as when it is used as local variable. Now I don’t want to have breakpoints in every constructor just to make sure that the debugger is working correctly. Need to check if that is still the same problem in 4.2 and if yes then I’ll fill a bug report.
Since a while I am bogged with a stupid debugger issue using XCode 4.0.2. So what gives:
In XCode4 you don’t have to declare your variables anymore if you declare them as properties (@property …). The compiler will take the info from the property definition and declare the variable. That works fine and the properties will show in the debugger when they are inside self or a member variable of self. So far so good.
But when I use the same class as a local variable the debugger is not showing that member. This is the code:
@interface ProgramStepEntity : BaseKVCMapper {
}
@property (nonatomic, retain) NSString* instruction;
@property (nonatomic, retain) NSArray* parameters;
@property (nonatomic, retain) NSString* target;
Here is another break point where the variable show up:
This can be fixed if I declare the variable:
@interface ProgramStepEntity : BaseKVCMapper {
NSString* instruction;
}
@property (nonatomic, retain) NSString* instruction;
@property (nonatomic, retain) NSArray* parameters;
@property (nonatomic, retain) NSString* target;
Now sure what is wrong but I sometimes see all the variable will show up and sometimes they don’t. It could be that the debugger is storing the list of variable once and if at that time the list is incomplete it will not show the rest.
– Andy
Object-C Categories: Why and How
Did you ever wonder what an Object-C Category is good for and how to actually code and use one? Here is a short rundown of why I needed one and how I actually did it including using a property in my category which is per se not allowed within a Category.
Cheers – Andy
GitoLite: Your very own Git Server on Mac OS X Server
GitoLite is a simple Git server that can handle user authentication per project / branch. Even though I don’t mind too much to shell out $300 bucks for a simple business plan with GitHub I want to keep my code closer to my heart that anything else. Even though I don’t think that my server is tightly protected it is at least my fault when something goes wrong. Lately there have been reports of nasty security breaches on user data servers and even Dropbox had their account unprotected for a few hours.
The first thing I was surprised of was the fact that there is no out-of-the-box git server available as it is with Subversion. Then I tried Gitosis but ran into issues with Python and abandoned that. Later I heard about GitoLite which gave me some grief at the beginning but eventually I could make it work out. The trick was to make sure that I was focuses on the task and made sure only to proceed if the previous step worked out.
Update: I copied this documentation to my XWiki.
Minibloq on iPad: Here We Come
Update: The project makes good progress and so I move this discussion over to my XWiki Page which handles screen shots, videos and code much better than this blog. That said I took time out of my schedule to setup gitolite server on my server so that the project can be shared (aka cloned) and later people can start to contribute back to the project as soon as I decided on the license. In case you are interested in the project you can send me your public ssh key and I so you will get read access to the project.
Update: Now that the Minibloq Alpha software is available for certain backers I could build and run it on Windows XP using VMWare Fusion. So I started to create an XCode 4 project for it and hope that I could compile and run it on XWindows on my Mac. In case that fails I might start on porting it to become a native OS X application using the OS X UI rather than XWindows. A few days ago I got the Ethernet shield but I am not sure yet that I can program the Arduino board with it.
A while I was able to contact Julian from Minibloq to discuss my idea to port Minibloq over to the iPad. He liked the idea and so I started to proceed with my idea. These are the next steps I will take to start the Minibloq on iPad project:
- Get an Arduino board -> Done
- Get an Arduino project up and running on my Mac (need to buy some parts) -> Done
- Figure out how to connect the Arduino Board to an iPad preferably using an USB cable -> It seems Ethernet Shield is the solution -> Received the Shield but did not find time to evaluate it
- Figuring out how to program an Arduino Board from an iPad
- Get Minibloq project to compile and run on my Mac OS X (as soon as the Alpha version is released on June 6th) -> Started the XCode 4 project
- Evaluate all dependencies of Minibloq and see if they can be used on the iPad and if not what it takes to port or replace
- Evaluate the GUI redesign of Minibloq
If everything works out I should have a good picture in mid June if Minibloq can be ported to the iPad and how long it would take. The final question after that is when shall we start with it because I don’t want to start doing porting while Minibloq is still under heavy development. On the other hand there might be many things like porting libraries or starting to design the GUI that can be done without having to do much catch up with Minibloq’s changes.
For now this was a good start and I am looking forward for the first release of Minibloq to get my hands dirty with it and to program my Arduino board.
By the way if any reader has any experience with connecting an Arduino board to an iPad (or any other method like with Ethernet or Bluetooth) please drop me a note on “schaefera AT me DOT com“.
This project could be a great after class program for my kids’ school where with an iPad, a cheap Arduino board and some electrical equipment the kids could create and program robots with very little technical knowledge. Eventually they will get that knowledge but with a lot of fun that comes way easier.
Cheers – Andy Schaefer
Blocklets, B-Squares, Minibloq, Arduino / DuinoBot and iPad == Kick-Ass Roboting
Lately I encountered some really cool projects on Kickstarter which are cool by themselves but together they could really make robotics a cool and fun activity. These are the projects I like and also back:
- Blocklets: A cool, lego-like building system with a lot of potential
- Minibloq: An open-source project to create a cool GUI to program and control robotic devices
- B-Squares: Solar-powered squares to produce, store and use the electricity
Now these projects are all pretty cool but together they could create a kick-ass environment for robotics especially in schools. Now if it would be possible to convert Minibloq to the iPad I can image that this would make it dead easy to program robots. Doing that might not be easy but I think this is worth while. If Minibloq can be made to work as easy as Garageband on the iPad then it could be fun, educational and easy to deal with robots and devices like Arduino. Using Blocklets to create structures and vehicles and B-Squares to power them would make it possible to create bigger and more complex robots.
A few years ago I bought a Lego Mindstorm box and even though it was fun for a while it is too complicated to program, does not work with a Mac and is limited to the sensors it supports. Even though I like Legos a lot I am not going to buy a new set of Mindstrorm and I don’t think that the user interface is good, fast or easy.
Right now I am trying to see if it would be possible or even feasible to port Minibloq over to the iPad. Then I would need to check if it is possible to connect an Arduino board to an iPad (I think it might work using an USB to 30-pin connector). After that I want to write a little test program that would access an Arduino board from an iPad to get a feeling how that might work before starting to code the iPad App.
Cheers – Andy
It’s Time to Cut the Cable
Since a few years I am trying to get away from the Cable Providers a little bit at a time. Initially I started using MythTV to get ride of the Commercials but unfortunately it takes way too much time to keep up to date, MythTV dropped the support for my TV card and it did not accomplish my goal. When the new AppleTV started to be sold I decided to dump MythTV and use AppleTV instead. Then I moved ahead and signed up for Netflix.
As of now I still need a Cable Provider because my kids want to see their shows and my wife want to see the local news. That said I am hoping that Netflix can ramp up their streaming offerings where eventually I can cut the cable and fire my Cable Provider. It is not only the money but I don’t see what service they actually provide to me that justifies the amount I pay. I know they need to pay royalties but then they don’t pay anything to PBS stations. They intentionally create the packages so that I have to buy way more than I actually need and they don’t want to provide a la carte selection of channels. Soon, I hope, the time will come where I can tell them to go to hell and take their outdated, consumer unfriendly system with them.
Since my kids took over the TV I did rarely watch any TV. Not only did all the TV series I was somewhat interested in either die or got so unbelievable stupid that I found playing games way more interesting. I think our family can live without the local news but my kids and my wife need to be able to watch their shows and they should either be easier to access or they should cost no more in total than the cable company.
Netflix streaming is not there yet but I hope it will be soon. That said considering that streaming plus 3 DVD out at the same time costs only $20 per month it still would save us $60 per month if we would ditch the Cable Provider. That would mean I could either buy some movies from iTunes or an entire series season from iTunes per month.
Well, I guess that my Cable Provider’s future in our home is in serious jeopardy and might be cut short soon.
– Andy
Halo Reach: Exodus with nearly All Skulls On On Legendary
Through the Halo Reach Daily Challenges I rediscovered the site of Tyrant which has a nice video walkthrough of Halo Reach with all Skulls on. This week the weekly challenge is to do Exodus with all but the Blind skull on. This makes it easier to handle because one knows the health status, sees the reticle and one sees the status of the jet pack, active camouflage or armor lock. Tyrant’s walkthrough is done with Blind on so it is easier to do in the challenge. So if you want to know how to beat it then use Tyrant’s walkthrough and I am not going to repeat it here. That said I want to give some additional hints that you maybe miss if you just look at Exodus.
When you start Exodus with all but Blind skulls on then you want to keep the following in mind:
- If you die, and you will, you can quickly hit Menu and select Save and Exit. If you restart the game you jump back to the last checkbox without having died
- If you die and you missed that short window (a few seconds) and the mission restarts you have to shut off the XBox otherwise you save the game when the mission starts. That said you loose any progress from the last saved checkpoint.
- In order to prevent to loose too much in the case you missed the Save and Exit time frame you should Save and Exit the game from time to time. This is also important if you start a new segment and you might end up in a situation w/o ammunition or other very difficult situation where the only way out would be to shut off the XBox.
- If you never tried this you might want to do it without Iron skull on because then you don’t need to Save and Exit too many times which speeds up the process a little bit.
- Be patient, make your shots count, crouch often and early. It is advisable to adjust your settings so that you can toggle between crouching and standing instead of keeping the stick pressed. But then again you need to adjust your muscle memory to avoid sticky situations.
- Conserve your ammo. “Count” the shots it takes to take out an enemy. This is important for high-value weapons like the DMR, the needler etc.
- Take cover early and often and peek just over the cover to take the shots. The shield only replenishes with melee and health only recovers 3 bars on each side from the lowest settings.
- In order to replenish you health with a Health Pack you need to have reduced health. You can stand in from of a HP and shot an overcharged Plasma gun or you can throw a grenade at a wall but just don’t die with that. Then you can pick up a HP while you have reduced health. Note: that the HP also replenishes the shield.
Now when you playing Exodus this is what you should consider:
- At the beginning get up to catch the Magnum rounds and then move quickly down the boardwalk. You don’t want to wait too long because the Skirmisher in the first fight would leave and you would need to use you Magnum rounds to fight the suicide Grunts right afterwards. If you kill on Skirmisher you can use its Plasma gun to kill the Grunts w/o using any Magnum rounds.
- After you called the Elevator on the Traxus tower head back to the first Kiosk and hide behind it. If you are lucky all brutes including the ones with the Gravity Hammer pass you by. But then they are a pain in the neck to find. If that happens stay on the lower level and use the sniper rifle to look for them. Also listen for any sound they make. If you hear them but cannot see them head back to the Kiosk and start all over again. Be aware that they throw grenades like crazy so you don’t have much time to kill them if they are close. The sniper rifle is pure gold to take them out. The second one with the Gravity Hammer maybe come down as well but can also be stuck near the Elevator. Slowly move forward until you know where he is.
- To fly from the New Alexandria root top to the landing pad on the other side I used the following tactic. Use long burst to go high (so hight that your are out of bounds) rather than short bursts but don’t empty the jets before you reached the other side.
- After you killed the big Kahuna inside the landing pad building you need to jump / jet up to the 3rd level immediately. Beside the fly over that was the point were I died the most times and there is no Checkpoint there until you finished off all the enemies.
- Here I would suggest to save the rockets for the Brute with the Gravity Hammer. Take out all the enemies inside the building and as many as you can take out on the landing pad. Make sure there is no Grunt around with the Fuel Rod Gun which would be a deadly mistake. If the Brute with the Gravity Hammer is the only one left inside the building take the Rocket Launcher and jump down to the second level. Expect some other enemies and if kill them or jump back up. If it is clear then engage the Brute and kill it with the Rocket Launcher (2 rockets should be enough). Then jump back up to the 3rd level and evaluate where the rest or your enemies are. BTW there is some ammunition on the other side of the building behind the 2nd crate. Not sure what happens when you jump out there so I wouldn’t try it until you are out of ammunition.
- Anytime you need to run quickly after you activated a button (calling the Elevator or activating the second Anti-Air canon) you should Save and Exit the game. If you get stuck but hit an checkpoint you might be stuck there forever and that would suck.
- Finally on the last run I was in a tight spot because after I activated the 2nd anti-air gun I lost some precious time. So when I came to the stairs I did not use the Moongoose to go up the stairs but exited at the bottom, activated the Active Camo and then hit up the stairs. If not the Grunts might see you can kill your Active Camo making you an easy target. When you are up to the table make sure you activate the button. If you activate it then the screen with get dark immediately otherwise you missed that and you need to retry right away.
Have fun and Enjoy that Challenge
– Andy
Unit Tests with iOS and XCode4
Since nearly 1 year I am developing iPhone applications but until last week I never found time to write and run unit tests even though in XCode4 it asks you if one want to include unit tests into the App. But it was just a question of time until I have to start using it. So one of my Apps started to fail over and over again because the web service calls either changed or just did not work. This became very tedious because I had to go through a ton of log files to figure out what didn’t work. Thus it finally was time to invest the time to create the unit tests so that the API is tested with little effort and I have a quick result on what is broken.
As usual the road to bliss was plastered with stepping stones, pitfalls and some frustrations which made it more difficult that expected but it did not prevent me from reaching my goal. These are the steps I had to take to make it work:
- If not already done create a new file of type Object-C test case class
- To begin with write a simple no-op test method with a NSLog statement so that you know when it is executed
- Go to the Project Configuration, select the test target, select all on the top and look for the group Unit Testing. There you set Test after Build to Yes.
- On the top left select the XXXTest schema and click on run
- To check the log please go to the log navigator (command-7) and select the top most log entry
- Inside there you look for Run custom shell script, select that row and hit the staple icon on the right. This will open the log view below.
- Now you can see if the test was executed
So far we made the test execute but if we try to test any of our classes we probably fail with a linker error. This is because XCode4 per se does not add any regular classes into the test environment except we already did that when adding the class to XCode. That said we can do that easily:
- Go back to the project configuration
- Select the test target
- Click on Build Phases
- Select Compile Sources
- Now we can add any implementation class file that we need to run the test. Please don’t add headers to this list (.h files)
- We might to repeat this a few times until we get all the necessary classes in
Now we can start writing tests that uses the classes we have. For now I only tested non-UI classes and I had to go ahead and use a switch in some of the classes to not use UI components otherwise the test would fail. I am not sure if there is a way to test UI components in the unit tests. On the bright side this forces developers to separate UI and service code more thoroughly which is not a bad thing either.
Cheers – Andy
iOS, Blocking User Input and CFRunLoopRun()
Yesterday I worked on new a Lab for my upcoming iPhone course and the topic was how to prevent user input while the application is gathering data from a server. Normally that is not a big issue when you just obtain data but that can be a big issue when you send data because then it could corrupt the server side.
So far I just created an UIView that was semi-transparent on top of the entire view preventing the user to click on anything below. But that did not work properly in every case because as it turned out the user input is not consumed but queued and handled as soon as the main thread becomes available. Therefore if I don’t disable the button or alike the code is executed again. This is the original code:
self.connection = [[[NSURLConnection alloc]
initWithRequest:request delegate:self] autorelease];
// Now show an animation
UIActivityIndicatorView *spinner = [[UIActivityIndicatorView alloc]
initWithActivityIndicatorStyle:UIActivityIndicatorViewStyleWhiteLarge];
UIView *window = [[UIApplication sharedApplication] keyWindow];
UIView *shield = [[UIView alloc] initWithFrame:window.bounds];
shield.backgroundColor = [UIColor blackColor];
shield.alpha = 0.5f;
[window addSubview:shield];
spinner.center = shield.center;
[shield addSubview:spinner];
spinner.hidden = NO;
[spinner startAnimating];
// Block the further execution until all data is received
CFRunLoopRun();
[spinner stopAnimating];
[spinner removeFromSuperview];
[spinner release];
[shield removeFromSuperview];
[shield release];
Soon I got the feeling that I block the main UI thread and so the input is not consumed until the system can execute it. After looking for a good solution I was more or less told that I should not use CFRunLoopRun() but let the main thread run its course and deal with the result when it comes in.
That said the transition was not easy but eventually I got all the callbacks in place and the thing worked fine except that the call was to short to test is properly. Thus I added a sleep inside the callback to give myself time to click multiple times but that did not solve the issue. Through debugging I then saw that when data are returned back from the server that it is still done in the main thread and so preventing the view to consume the user input.
This is the code afterwards:
// 1. When the call comes in and we start the loading
self.spinner = [[UIActivityIndicatorView alloc]
initWithActivityIndicatorStyle:UIActivityIndicatorViewStyleWhiteLarge];
[self.spinner release];
UIView *window = [[UIApplication sharedApplication] keyWindow];
self.shield = [[UIView alloc] initWithFrame:window.bounds];
[self.shield release];
shield.backgroundColor = [UIColor blackColor];
shield.alpha = 0.5f;
[window addSubview:shield];
spinner.center = shield.center;
[shield addSubview:spinner];
spinner.hidden = NO;
[spinner startAnimating];
// Now execute the call to the web service
self.connection = [[[NSURLConnection alloc]
initWithRequest:request delegate:self] autorelease];
// 2. When the all the data is received we do:
[spinner stopAnimating];
[spinner removeFromSuperview];
[shield removeFromSuperview];
// Call back to the service that we received the data
[self.depotService handleResult:root];
Finally I went to the backend web service and made sure that handling of the call is delayed. Then I was able to test it and voila it worked like a charm. I could click multiple times on the button and when the shielding view went away the user input was consumed and did not cause more calls to the server.
There is still some stuff I have to learn how things are done in iOS in contrast to backend Java as I am used to. On the other hand it forces me to understand the underlying concept in details which will be a great asset when given the class.
– Andy