Archive for the ‘Storage’ Category

Custard On Top

September 24, 2013 Leave a comment

I’ve started a new IBM Redbook residency this week. A bit different to those that I’ve previously worked on, as work and family commitments don’t allow for travel. So I’ll be working remotely, and part-time, from Australia while the rest of the team are in Mainz, Germany.

The aim is to produce a new Redbook on IBM N Series Clustered Data ONTAP (cDOT), and to update the existing N Series hardware and software guides. I was also on the team that produced the previous update to the N Series hardware and software guides.

I’d describe myself as very experienced with Data ONTAP 7 (or DOT 8 in 7-mode). I was first NCDA certified in 2005, and have since collected an embarrassing array of NetApp certs. I also worked at NetApp as a technical instructor, and wrote the original NCDA certification study guide. But so far my experience with Clustered mode has been minimal.

For me, this is a great opportunity to keep my technical skills current, whilst contributing my experience and new knowledge back to the book.

I’ll update this post over the next few weeks to document my experiences.

Week 1 – Back to school

The first week was spent in a Clustered ONTAP 8.2 Administration course, to get us all up to speed with the new features. The course was presented via WebEx in a “virtual live” format. The WebEx format worked well, with a very knowledgeable instructor, and plenty of opportunity to ask questions.

The gotcha for me, is that the course was scheduled for Central European time, since that’s where the rest of the residents are located. That’s 5pm-1am here in Melbourne, Australia. With my day job, and family too, this made for a _very_ long week.

My first impressions of cDOT are very positive, with much of the traditional ONTAP goodness still familiar, though enhanced in many ways. At the same time, the change from Active/Active ‘Clustered’ controllers, to a scalable cluster that is built from ‘HA Pairs’ has altered the behaviour of many features.

I felt a bit like a wide-eyed Dorothy in the Wizard of Oz – we’re not in Kansas anymore!

Toto, I've got a feeling we're not in Kansas anymore

Here’s a quick list of some changes in Cluster mode:

  • Clustered ‘HA Pairs’ replace the traditional dual Active/Active controller design.
  • Fail-over within a HA Pair is improved by using Storage Fail Over (SFO) instead of Cluster Fail Over (CFO).
  • Different resource types, such as aggregates, volumes, and network ports & interfaces,  are now bound either to physical nodes, vservers, or are cluster-wide.
  • Vservers, otherwise known as Storage Virtual Machines (SVM), replace the vfiler Multistore feature.
  • Whereas a 7-mode Virtual Interface (VIF) is bound to a physical node, its replacement, a Logical Interface (LIF), is bound to a logical vserver (cluster-wide).
  • SnapMirror loses its Qtree and Synchronous modes, but gain several other features, such as LoadSharing mirrors
  • SnapVault is now based on Volume SnapMirror, so retains the storage efficiency features of the source (deduplication and compression)
  • Some features from 7-mode are not available, but may be expected in future versions. (e.g. SnapLock, MetroCluster, SyncMirror, etc).

There are many more that I’ve not mentioned in the above list, but for those you’ll need to read the book (or at least the updates to this post 🙂

Week 2 – Meet the team

It’s only the start of the week, but here’s a screen shot of the kick-off meeting. We used Google Hangouts as a good way for the team to ‘meet’.

My fellow residents are (from left to right): Christian Fey, Danny Yang, Michal Klimes, Roland Tretau (leader, main screen), myself, and Tom Provost.

Roland TretauResidents

This meeting was a brain storming session to create a draft Table of Contents for the book, main parts, chapter topics, etc. Once the ToC is agreed we’ll assign different chapters to each team member to complete.

I’m going to start on the updates to the existing H/W guide and 7-mode S/W guide. This is a good fit for working remotely as it’s separate from the Cluster mode book, though there will be common sections that should be easy to adapt. Once I’ve finished updating the existing books I’ll rejoin the rest of the team.

All Redbooks are developed in Adobe FrameMaker, so one of the first tasks for any resident is to install the FM software and the ITSO custom toolkit. The ITSO have developed an applet called QAM (Quick Access Menu) that automates many book creation tasks. This also ensures that the authors, who are often new to FrameMaker, always follow the ITSO style.

Mock up of the cDOT cover

By the end of the week I’ve merged the contents of both the h/w and s/w guides into the new book files, and have started reviewing new material for inclusion.

The main challenge so far hasn’t been working remotely, but in trying to find time to work on the book part-time. I’m still doing my day job too, and anyone who’s worked at IBM knows that already requires 100% effort!

Week 3


Week 4


Week 5


Week 6


Categories: Storage, Training

Bus timetables and storage efficiency

March 28, 2011 1 comment

Sometimes storage insight comes from the most unlikely of places…

I recently attended an IBM IT Architect networking event where the topic was IBM’s Smarter Planet initiative. One of the talks, by IBM software Architect Andy Heys, was on the topic of Smarter Transportation. His talk included an example of IBM’s Traffic Planning Tool and how IBM had developed a traffic prediction model for the Singapore Land Transport Authority (LTA).

This gave me an unexpected insight into the parallels between Singapore bus timetables and storage thin provisioning. Seriously!

The bus bit

The story goes, that after having developed a system to accurately predict traffic flow for up to 60 minutes into the future, IBM then suggested taking the next step, which was to start adjusting traffic signals so as to reduce congestion. It’s a natural assumption that studying and predicting traffic flow would then be used to improve that same traffic flow…


At this point, says Andy, “we realised that they had more insight into the problem than ourselves”. The LTA was more interested in improving bus arrival time prediction than in directly improving traffic flow. That’s because they understood that any improvement in traffic flow would simply attract more traffic to the roads, which would rapidly return them to the same congested state, and probably worse.

Their plan was to attract more people onto public transport by reducing the schedule variance and unpredictability of bus arrival times. After all, who wants to wait at a bus stop if they are unsure of when the next bus will arrive. More people taking the bus would also reduce the number of cars on the road, and indirectly improve traffic flow too!

The storage bit

At first glance, traffic prediction has some obvious similarities to storage capacity planning – measuring utilisation and plotting trends. But I think the more interesting parallel is the competition between high storage utilisation (ie. road congestion) and storage efficiency tools such as thin provisioning, compression and deduplication (ie. methods for improving traffic flow).

Just like with traffic planning, storage also suffers from unexpected side effects, such as how increased available capacity (eg. via thin provisioning, or deduplication) is rapidly consumed by newly attracted workloads. With the same end result being a larger and more difficult problem to manage.

So, what is the analogue for public transport in the storage world? How do we improve capacity management, not by increasing storage density, but by reducing the storage burden?

It’s at this point that the conversation changes to be about data archiving, which serves the dual purpose of managing capacity and “getting cars off the road”.

Of course, I’m not saying that storage efficiency tools are unimportant, just that they are not enough on their own. They provide great value by decreasing (sometimes dramatically) the amount of physical storage that is required.

Back to the buses

The difficult question then is “how to encourage good behaviour (public transport / archiving) when the demand is always for more cars (increased storage capacity, greater performance, lower cost, and faster provisioning)”?

To use another Singaporean traffic analogy, this is already a solved problem, where congestion charging is used to toll traffic within the CBD area (ie. Tier-1 storage).This may never be a popular tool, but it is certainly an effective one. Tolls (and archiving) are like broccoli – good for other people 🙂

I suppose that I should highlight that I’m not really advocating for charge-back, but rather for intelligent capacity management.

While it might be difficult to introduce chargeback for an existing storage environment, might it be possible to introduce new capacity, capabilities, and costings which encourage proactive storage management? If it’s easier, cheaper, and faster to catch the bus rather than to drive a car, then more people will choose the bus willingly.

The (unfortunately very dated) diagram below shows the impact of congestion charging on Singapore CBD traffic over a 20 year period. I like how similar this looks to the archetypal chart for storage growth (always increasing), except the doubling period is about ~18 years, not ~18 months as for storage (YMMV).

Effects of ALSData is scaled to 100% for base year 1975
Blue: Car Population, Yellow: AM Inbound, Red: PM Inbound

Can a similar model of selective charge-back (or relative “charge-less” for lower storage tiers) work in enterprise storage? It always amazes me that it’s easier to impose tolls on public roads than on commercially managed storage. For many environments which don’t practice storage charge-back, perhaps the answer is simply to make the “public transport” option more appealing?

PS/ A sub-title for this post might be “Is public transport (ie. archiving and/or tape) dead – no way!” 🙂

Categories: Storage