Analyzing Hadoop’s internals with Analytics

Interesting post by Richard Mc Dougall, VMware CTO (thank you)

As part of our Big Data efforts, we have a team focused on Hadoop that is working hard to ensure Hadoop runs well on vSphere. We published a paper last year on Hadoop performance, and have a lot more in the pipeline.

More recently, I took up a challenge to see how much we could learn about Hadoop I/O in a very short time, using our dynamic tracing framework. The results were quite interesting.

To ensure I position this work correctly, it’s really a work-in-progress study that I’d like to engage a discussion around.  It’s what we’ve learned by digging into the architecture, to help us make engineering decisions. I hope it’s helpful, and really want to get your feedback, so we can steer future iterations of these investigations.

The data in this post is a lead up to some simple modeling of Hadoop in a cluster, so we can rationalize how Hadoop will scale in different topologies.

Read on here

Maximo Integration Framework- Error Extract Functionality

With the Maximo base services v7.1.1.4 fix pack, the integration framework data import and data import cron task functionality now provide a mechanism that enables Maximo users to identify the error management mechanism of their preference:

*GUI-based using the Maximo Message Reprocessing application
*File-based using the Maximo error extract mechanism

File-based error management relies on reprocessable error files that you can now download from Maximo and modify using a file editor of your choice. The Maximo integration framework inbound processing allows you to reimport these reprocessable files to Maximo without requiring any structure changes.

The attached document contains a detailed description of how to take advantage of Maximo integration framework error extract mechanism.

If your integration framework configuration includes an instance of the XMLFILECONSUMER or FLATFILECONSUMER crontask, please follow the configuration updates listed bellow while applying the v7.1.1.4 fix pack:

1) Deactivate all instances of the XMLFILECONSUMER or FLATFILECONSUMER cron tasks
2) Apply the v7.1.1.4 fix pack
3) Create new instances of the XMLFILECONSUMER or FLATFILECONSUMER cron tasks using the same parameter values that existed on the deactivated instances.
4) Reload the new cron task instances via the “Reload Request” action menu option.

More details here

“Xtrem” Thunder in the Forecast for EMC?

Post by Scott Dietzen (thank you) from Pure Storage

The rumors have been confirmed—EMC (NYSE:EMC) is indeed acquiring XtremIO. Like Pure Storage, XtremIO is crafting a next-generation storage array, purpose built for 100% solid-state flash. Gartner recently forecast that this all-flash array market is going to reach $4B by 2015. It’s not at all surprising that EMC wants a piece of the action. Our expectation is that the XtremIO technologies will eventually get folded into EMC’s Project Thunder (announced earlier this year for delivery in 2013), and/or serve as the basis for an entirely new all solid-state array.

Read on here

IBM System Storage SAN24B-5 documentation

IBM System Storage SAN24B-5 Switch Installation, Service and User Guide.

Get the guide here

IBM System Storage SAN24B-5 Switch Quick Start Guide

Get the Quick start guide here

SAP with IBM Tivoli FlashCopy Manager for VMware and IBM XIV and IBM Storwize V7000 storage systems

This white paper describes IBM® Tivoli Storage FlashCopy ManagerTM for VMware® with SAP® NetWeaverTM and provides a complete data protection solution for VMware vSphereTM environments which can be administered from the VMware vCenter console. FlashCopy Manager allows taking near instantaneous backups of VMware data stores by utilizing the FlashCopy functionality included with the IBM Storwize V7000TM, IBM System Storage SAN Volume ControllerTM, IBM XIVTM and IBM DS8000TM storage systems. Protection jobs with Tivoli Storage FlashCopy Manager for VMware functionality can be created through the Tivoli Data Protection VMware vCenter plug-in.

Get the Whitepaper here

99 Notre Dame, San Jose

Good post by Hu Yoshida (thank you) on some memories around the RAMAC etc.

This morning I accompanied a young person to the Superior Court in San Jose. This Court is located at 99 Notre Dame in San Jose. Once we went through the metal detectors we joined a long line of people in a crowded waiting room. What surprised me was the display along the south wall, which reminded me that this location was the birthplace for magnetic disk recording. IBM established a research lab at this location in 1952. The first lab director was Reynold B. Johnson and the first Random Access Method of Accounting and Control, RAMAC, was announced here in 1955.

Read on here

EMC buys XtremIO for $430m

Post by Shmulik Shelach (thank you). So finally the rumor is a fact :-)

The storage systems company is EMC’s sixth acquisition in Israel.

After several months of intensive talks, EMC Corporation (NYSE: EMC) is acquiring storage systems company XtremIO for $430 million. “Globes” was the first to report the imminent acquisition several weeks ago.

XtremIO was founded by a group of Israeli high tech veterans including Aryeh Margi, a co-founder of M-Systems; Shuki Bruck, Yaron Segev, and CEO Ehud Rokach, a former senior executive at Orckit and CEO Corrigent.

Read on here

PowerVM Active Memory Expansion (AME) and Oracle Database: Experiences and Best Practices

This paper provides an overview of Active Memory Expansion (AME) and documents two scenarios where AME was successfully exploited with the Oracle Database Server hosted on a Logical Partition (LPAR) on a POWER7® system. The paper also describes best practices that should be followed when deploying AME.

Get the paper here

Introduction to Storage Area Networks and System Networking

Introduction to Storage Area Networks and System Networking

The plethora of data created by the businesses of today is making storage a strategic investment priority for companies of all sizes. As storage takes precedence, three major initiatives have emerged:

Flatten and converge your network
IBM takes an open, standards-based approach to implement the latest advances in today’s flat, converged data center network designs. IBM System Networking solutions enable clients to deploy a high-speed, low-latency Unified Fabric Architecture.

Optimize and automate virtualization
Advanced virtualization awareness reduces the cost and complexity of deploying physical and virtual data center infrastructure.

Simplify management
IBM data center networks are easy to deploy, maintain, scale and virtualize, delivering the foundation of consolidated operations for dynamic infrastructure management.

Storage is no longer an afterthought. Too much is at stake. Companies are searching for more ways to efficiently manage expanding volumes of data, and to make that data accessible throughout the enterprise; this is propelling the move of storage into the network. Also, the increasing complexity of managing large numbers of storage devices and vast amounts of data is driving greater business value into software and services.

With current estimates of the amount of data to be managed and made available increasing at 60 percent per annum, this is where a storage area network (SAN) enters the arena. Simply put, SANs are the leading storage infrastructure for the global economy of today. SANs offer simplified storage management, scalability, flexibility, availability, and improved data access, movement, and backup.

Welcome to the era of Smarter Networking for Smarter Data Centers!

The smarter data center with improved economics of IT can be achieved by connecting servers and storage with a high-speed and intelligent network fabric that is smarter, faster, greener, open and easy to manage. IBM System Networking solutions.

This book gives an introduction to the SAN, Ethernet Networking, and how these help achieve a smarter data center.

WD bigshots spin superfast disk roadmap

Post by Chris Mellor (thank you) over at El Reg

Western Digital’s disk drive roadmap has hybrids and tech transitions coming to shrink I/O latency and regain fast areal density growth.

At an exec summit in Vienna on 8 May, WD discussed coming hard disk drive (HDD) technologies without pre-announcing products and with some unattributable thoughts around hybrid flash and hard disk drives. Conversations also covered shingle magnetic recording and EAMR (energy-assisted magnetic recording), which will hopefully enable the industry to regain a 40 per cent compound annual growth rate in areal density – it has currently slowed to 20 per cent as perpendicular magnetic recording (PMR) matures.

Read on here

Follow

Get every new post delivered to your Inbox.

Join 370 other followers