Even more WattDepot

After my previous post where I reviewed another team's project, I found myself working on actually enhancing the system.  The additional commands to be implemented were:

 

1. set-baseline: gathers energy consumed in hour intervals over a period of 24 hours to use as a baseline for further comparison

2. monitor-power: prints a timestamp and the current power continuously in user specified intervals

3. monitor-goal: this monitors the power and compares it to the baseline set from the first additional command.  A user picks a percentage reduction which monitor-goal will use when comparing the current power to the baseline to see if the reduction is sufficient.

 

We were able to finish all the new commands and have them implemented,  I believe as of this point, there are still a couple of bugs in the system but they are minor and can be fixed without much trouble (is what they always say right?)  But overall, each command works and the system compiles.  When looking at our enhancements with regards to the Three Prime Directives (you remember them from the previous post right?)  it is my humble opinion that they still fulfill them.  Our enhancements have an obvious benefit for the user, especially monitoring by goal.  We continued to add comments and test cases for each new enhancement meaning development can be continued smoothly.  Issues were created and fulfilled letting everyone know who to contact for what part of the system.  The wiki pages were updated to reflect the enhancements. 

 

One thing that I want to mention briefly is how surprisingly difficult it can be to work with someone else's code.  In their design, an instance of each command is created every single time you type something in to the command prompt.  Also, each command is supposed to return a boolean value determining whether or not the command ran successfully.  This was fine for the previous commands, but once we had to add in set-baseline, which stored data somewhere, things had to chagne.  At first I wanted to just use a normal hashtable to store the data by tower.  But I realized first that under the current design, I couldn't return a hashtable or anything because it wanted a boolean.  And then, because it remade each command every time, I was unable to store the hashtable in the command.  In the end, I had to go a step above the commands and store the hashtable there.  I wanted to just send the hashtable as a parameter to the command function, but I couldn't because then any command could access the inner content of the hashtable and mess with data.  While it may be fine since I want to change the hashtable anyway, it's bad practice to allow such things and I had to change my own design.  It turned out to be a simple matter of making an encapsulating class for hashtable and then passing it, along with some get/put methods in to work with the hashtable without actually having it. 

 

While this seems like a simple problem at first, because I was working with someone else's code I was not immediately aware of the specifics of the system.  Therefore, it took me a while to figure out what was wrong and why.  Of course once I did, it seemed very obvious, but it still took longer than it would have had I been working with my own design the entire time.  But learning to do these kinds of things I feel is very important because as systems get bigger and bigger, it becomes impractical for a single person to work on everything.  So it's good practice for me to be working with someone else's system and learning to think like they do.

 

Code Review

After having a chance to work on my own team's Hale-Aloha-Cli, I now have a better understanding of what it's supposed to do and the methodology involved in doing the project.  As an exercise in looking at other people's code, I went through a technical review of another team's implementation of the same project using the same sort of techniques.  To be more specific, they are under continuous integration and are managed through issues.

I touched upon those topics briefly in the previous blog, but for those too lazy to read again, or for the first time, here's another description of it.  For those who know this, skip on down to see yet another overview of the project and then the actual review of it.  So continuous integration means that the project is worked on at all times.  This seems pretty self explanatory but it has a couple of major problems and solutions involved.  First, because multiple developers are constantly working on the same source, if two people download something, work on it individually, and then try to upload it, there may be some errors in each other's additions due to the concurrent development.  So we have a system that will check additions, tell us if there's a difference we need to look at before uploading, and tell us when our version isn't the most up to date.  Therefore, continuous integration.  To get something like this to work, it involves having everyone upload pretty often, because the bigger a change is, the harder it is to make sure everything works with it.

To go with that we used issue based project management and google project hosting.  We split the project into small pieces that we could do in a day to keep things running smoothly. Ideally, we'd have enough issues for everyone to work on something at all times without getting bottlenecked.

So! On to the review.  Almost.  WattDepot, as you hopefully know, is a project for gathering, storing, and manipulating data taken from energy sources.  It's currently being used in some of the dormitories at the University of Hawaii to keep track of how they are conserving energy.  Also, students are shown some graphs and figures, and are given incentives to save energy and so far, it seems like the project has been a success.  The project we and other teams were working on is a command line interface for the WattDepot.  So each implementation should feature a command line and a couple of useful commands for accessing data gathered by WattDepot.  They are:

help: provide a helpful message explaining the usage
current-power: returns the current power
daily-energy: gives the energy given in a certain day
energy-since: gives the energy consumed from a certain date to the most recent data
rank-towers: order them by energy consumed.
quit: exit the system

To evaluate any program, I mainly look at what are considered the three prime directives of any piece of software.

Prime Directive #1: The system successfully accomplishes a useful task.
Prime Directive #2: An external user can successfully install and use the system.
Prime Directive #3: An external developer can successfully understand and enhance the system.

Here is the system I am reviewing.
For the first directive, I downloaded the latest distribution here.  Everything compiled correctly and I was able to run the program.  I tried each of the commands and they all worked for the most part.  Rank-Towers sometimes didn't work 100% properly, but as a whole, the project did a good job at accomplishing something useful.
For the second directive, I at first felt that I could immediately say they did it, because I was able to compile and run everything.  But I'm very familiar with how it works and what it's supposed to do.  For an average user, this may not be the case so I looked at their project page to see if there was at least an overview and instructions on usage.  Both were found, as was a jar file so users will be able to use the cli even without a JDK.

Lastly, I had to check the documentation and structure of the project to determine if an external developer can successfully understand and enhance the system.  On the project page, there is a developers guide which gave clear instructions and guidelines on how to use the system.  They also included javadocs, which I was able to easily compile and build which gave further information on each part of the system and all pertinent methods.  Finally, the code itself had a lot of comments which made it easy to see the logic behind the development.  The structure was modular, so I could make a new command without breaking anything else, which makes further development easier.  Each class had an associated test class to ensure that further development won't break something (or if it does, you will immediately be aware of it because the test will fail)  Some parts did not have 100% coverage, but that's mostly due to some try/catch statements that are never caught because they always work correctly or handle an error before reaching the catch. 
In addition to having good documentation and instruction on how to develop the code, which developer was responsible for which part of the project was clear due to the issue based system that google hosting provides. By looking at which issues were completed by who, I could easily tell who I could ask for help if I didn't understand a certain part. Interestingly to note, of the three developers, two did the brunt of the work while the other kind of slid along.  While this isn't really relevant to the directive, it makes me wonder if maybe he was in charge of divying out tasks or some other effort not immediately obvious, or if he was just lazy.  Overall though, the system is clearly able to be enhanced easily.

And with that, the program fulfills all three prime directives.

Issue-based Project Management

So once again, I found myself working in a group. This time we were developing using issue-based project management under continuous integration. In other words, we tried to split the project into smaller issues that could be completed in a couple of days.  The point of this is to allow everyone to always be working on something without being bottle necked waiting for others.  Also, because issues are small, it's probably easier to not get discouraged looking at the size of the project.  Finally, by splitting everything in to manageable little pieces, we were able to clearly think about how to best split things and keep everything separate and testable.  Because we needed to make sure that everyone always had something to work on, we had to make sure that what we were doing was very modular. Then, because we were doing everything concurrently,we used a project host and continuous integration to make sure that everything kept working and we didn't bump each others' work out.  The hardest part of this for me was keeping up daily with working on issues.  I tend to like to sit down and work all day and then relax afterwards, but dividing things up requires a little work per day over a longer period which is something I still need to work on getting used to.

Getting to the actual project, we were working on a command line interface for use with energy data collected from the Hale Aloha residence halls at the University of Hawaii's Manoa campus.  It can be found here. The project involved creating a simple shell and then making various commands to interact with the energy data given.  We were able to successfully make commands for ranking towers, finding the energy since a certain time, finding daily energy and the current power. However, we had to hard code each command in to a list.  So any additional commands have to be added to that in order to work.  So it wasn't the most elegant solution, but it got the job done and did it in a modular fashion.  New commands are pretty easy to create, and the processor and main interface probably don't have to be touched.  As far as documentation goes, I think we did alright, although because I was working on the project and had a pretty in depth understanding of what we were doing, some things that I thought were sufficiently explained may not have been.  This is where I see the importance of testers who aren't directly involved with the project.

Overall, it was a good experience, and something that I think I need more practice in.  Working with people can be troubling but makes things so much better if you're good at it. <- main lesson for the day

WattDepot

Continuing on with the electricity and energy rant from the previous blog post, I started doing some katas on something called WattDepot.  It's a program that centralizes a lot of different monitors and sources to easily look at data and keep track of things.  Because the main program doesn't really care about what the actual monitors use, someone like me is able to connect to the servers and start working with data even though I don't even know how they're getting the information, which makes it really convenient for a lot of reasons.  Not being stuck to one standard that a company decides on is good and lets developers focus on other aspects. Like katas :P

They actually took longer than I thought they would.  There were about five and it took me at least five hours.  But the time spent on each one wasn't equal, and I took breaks. 

The first kata asked to merely print a list of sources that the server is getting information and give a description of them, which is also conveniently provided.  Not only that, there was a sample program with the distribution of WattDepot which showed the basics, basically, I didn't have to do anything for the first one. 

The next one asked for some data manipulation to determine latency based on the given timestamp which was a little more complicated but not too much.  I tried to make a helper class implementing comparable, which seemed to work ok but i'm not sure if there's a better way to do it easily. One thing interesting here though, is that this is basically my first time working with a server that doesn't always transfer in the blink of an eye.  So the way I had it set up first, with a bunch of nested loops, took forever to do and really made me think about how to structure my code to get things working as smoothly as possible, I'm sure i didn't do as well as i could have, but my end results were certainly better than what i started with.  You don't really notice these things when it's the difference between a fourth of a second and a fifth, but because the server was bottle necking things, it helped me out.

The third and fourth were just more complicated forms of the second problem, involving using time and averaging or looking at a bunch of data points.  Again, they took a long time to finish, so instead of just leaving things blank until the output spewed out, I put little messages telling you where the program was, which is something I haven't really had to do before.

The fifth gave me some problems because it involved knowing when the last monday was, given only the current date! Turns out it's a pretty simple problem of using java's implementation of a calendar.  The only tricky thing is that the months seem to go from 0 to 11, so for the longest time, the calendar was giving me the wrong day and i didn't know why.  The day of the month (8th) and the month itself (11th) looked ok, but only when i made it print the actual name did i see that 11 was equivalent to December.  After I figured that part out, the rest was pretty simple, though I saw some areas where i could make my code more efficient.

Overall, these katas were a good learning experience for more real world tasks where things don't just finish super fast.  I haven't really had much practice with optimization and structure so it was especially helpful.  Also, being able to so easily pull up data on the energy and power usage of an actual set of buildings was pretty cool, and probably also helped keep me more interested in katas.  They're pretty fun, and if you want to look at the program, here it is.

Of course, looking at the API for the thing, I see that there are a lot more of advanced features that I didn't even touch, but from a beginner's point of view, even just this is pretty good. That's all for today.

Energy

It's kind of strange, but until this point, I didn't really know much about energy or conservation or anything.  Of course I knew the basic stuff like "it's good to recycle" and "turn off the lights when you go out" but that was about the extent of my interaction with energy conservation and usage.  Of course, when gas prices go up, I silently complain to myself, but just feel resigned to the fact that that's the way things are.  But as it turns out, that's not exactly true.

Due to Hawaii's geography, it turns out we have some challenges and benefits that most other places in the world don't share.  For example, we're severely lacking on space so it's impractical to create huge power grids, but we have almost all types of natural energy things like wind, waves, lava, etc.  Also, we spend a ton on gas and use way more than a lot of other places on the mainland USA.  So it makes sense that Hawaii would take the initiative in trying to find clean, reusable energies.

But another interesting thing is that in the pursuit of reusable energy, there can be some harm to the environment as well! So you can't just say "hey guys, let's put up windmills wherever possible and solar panels everywhere else.  Windmills mess people up with the noise and sight and have a bad habit of killing birds.  Solar panels have very little maintenance but aren't nearly as effective as other means, which means that we'd have to use a ton to get the same effect, covering a lot of land we may want to use for other things.  With just those two examples, I began to see that the problem really isn't too simple. But the thing that most excited me was the fact that data and knowledge was such a big deal to people.

Personally, i know that computers drain their fair share of electricity and that leaving things on, or building a power hungry system can really have an effect on how much you pay every year.  And I know there are tools out there to monitor how much you're using, but I never got around to buying one so I just went on living my happy, ignorant life.  But it turns out people are interested in knowing these things, for one, by looking at how much you're using, you have a concrete reference with which to improve.  Also, when power companies look at this sort of data, they can spread the load and make sure that there is enough electricity at peak times and maybe a little less when there is less need for it instead of just supplying the same amount all the time.

To that end, there are a lot of people trying out pilot programs and other things to get the community more involved in conserving electricity and moving away from fossil fuels.  I think it would be really cool if Hawaii could be totally independent, at least in energy if not in food or other commodities.  But there's certainly a lot to be done to get even a little closer to that point, and I think one of the main things is public awareness.  Most people don't know, and don't care.  If they don't see how something effects them directly, they aren't really interested in it, which is why things that get them involved somehow will probably work better (like giving me free monitors :P)

Training and Review

When I first started writing code, I just wrote whatever I wanted, without regards to format or style.  If something wasn't working, I just figured it out myself without ever asking someone for help. And even though I always wrote simple programs (usually homework) I often found myself stuck for a long time only to find a simple little error that I could change in less than a minute.

Not only did I do things by myself in whatever rough, ship-shod way I wanted, I never practiced anything . I just started straight off on my programs, sometimes having to redo almost all of it because I realized that there's a huge logical error in the way I structured something.

As things became more complex, I found myself thinking about what I was going to code and practicing small little things first before putting it all together.  And then when I had trouble I found myself going to others for advice or help. Because suddenly others are involved, I needed to start thinking about how to style, review, and distribute what I wrote.  Here are a couple things I (slowly) began to realize.

When you write something, it's important to review the code in addition to actually testing it.  Testing is good for finding defects or bugs, but often I would write a program, check it for bugs, and then call it a day, only to find out that I forgot to implement something important.  By reviewing my own work, I was able to make sure everything that needed to be in there was.  In addition, I was able to cut down on actual coding time because with things planned out and reviewed systematically, I didn't have to rewrite as much as I normally would. And then later, when I reviewed things in group projects, it ensured that everyone had some idea of what was going on with the program.  (instead of having one person do all the work and the rest sitting idly.)

So when you test, you can also review what you're going to test! For example, in a previous blog or two I was talking about my robocode project.  In testing it, I really had to think about what to test.  There were no super obvious defects because the robot run and generally shot at the enemy.  But by first making tests for each helper function, I was able to ensure that they gave the correct output each time (and I found that they didn't!)  Then I made some behavior tests to make sure the robot did what it was supposed to every time.  Like for some reason, my robot would always miss when robots stopped to turn and then started moving again.  It thought of them as stationary targets and fired a wave of bullets, all of which missed once the enemy moved just a bit.  By testing behavior, and things like movement or firing, you can further ensure that your robot works correctly and this requires some review of the tests.

Also in terms of reviews, there are multiple "techniques" or "forms".  The one that I most utilized was a "code walkthrough" which involved me asking people to look at my code and tell me what's wrong.  Or I'd say I'm interested in implementing something and do they have any ideas on how to do it.   It's quick, it's easy, but a lot of times people wouldn't be able to help. No harm in trying though right?

Another form that I found used often in programming classes is technical reviews. They involve looking at code for defects, and then noting it or writing something down that the writer can look at later.  Because these required knowing what to look for, and usually forms to fill out, it was more time consuming than walkthroughs but probably helped me out more.

And lastly, something that I haven't actually tried, but have heard of is inspection.  This one seems the most time consuming but most useful of all.  It involves planning things out, forming teams, looking at objectives, making comprehensive lists of errors, fixing them all, and verifying that they've been fixed.  It involves multiple people and a lot of planning which makes it troublesome to do in a classroom setting. And because we're usually writing simple things, it may be overkill.  But out in the "real world" I feel that it's really useful.

In addition to reviews, I used a build system to get all the files together when I distribute some programs.  I found it really useful, but using build systems was something new to me.  Also, the programming theory and styles was a lot different compared to writing in something like Java, or C.  In those languages, I wanted to waste as little space as possible, by not including things twice, or double declaring variables.  But in a build system, I found myself adding dependencies to everything I needed, regardless of if something else also depended on it.  The reason for this is that if someone has part of the files they need but not the others, the necessary dependencies may not run.  Also, build systems are cool and don't rerun things you've already done so it never hurts to just go wild with dependencies and make sure everything required goes on every target or function.

Finally, moving away from reviews and going back to how I started practicing simple things then adding them together, is the concept of a kata.  In Karate, a kata is a formalized set of movements and techniques that you can do over and over to refine and perfect.  In computer programming, it's a training exercise designed to help the learner gain some understanding in to concepts and make/fix mistakes without having an actual cost to it (like it would if you made mistakes when developing some professional software)  So by doing simple little katas it becomes a lot easier to make something more complex, something I especially noticed when writing my robot.

That's all I can think of for now, and really I've probably written too much already. Till next time then.

SVN and Project Hosting

Amazing. I don't have to put everything on a thumb drive and transfer it here and there?  I can have others help me with programs without having to worry too much about syncing? Life is good again. (slight exagerration)

But today I was finally introduced to online project hosting and configuration management.  It was really easy to set up using google project hosting and I found how useful it is already.  I began working on it on my laptop, which I usually carry around everywhere, but today was too lazy to take out.  Instead of having to transfer the files or keep a copy stored somewhere, I was able to go online and download the source from the project host.  While I could do the same with any online file host site, because it's an svn project, I can ensure that changes are tracked and the directory is configured correctly without having to worry about zipping things here or there.

The only thing I found that might be a problem is that to add collaborators they must have a google account. This isn't really that big a problem since most people have one or can easily make one, but it might be nice if there were a way to allow anyone to collaborate.

For those interested, the project I made just contained the robot that I was working on earlier in previous posts. Here's a link