Data Marting and BI Development: Part 2

So when we talk about Data Marts and Data Warehouses we're really talking about decision making systems. These software systems provide business decision makers with the data necessary to make well informed decisions.  This may take the form of customer data, performance data, sales data, etc.  All of these things support business decisions and strategies.  The main difference between a mart and a warehouse is that a Data Mart is only concerned with a single subject matter area.  So from my above list, we'd create a Data Mart for sales, another for internal performance indicators, and yet a third for customer data.  All of these marts together would constitute a Data Warehouse.

Now following the previous example and applying it to our current environment here at work, we'll be producing multiple "Data Marts" for each subject area in the company.  I use the quotations to indicate that what we're doing is not traditional Data Marting, but a hybrid approach that will leverage SharePoint as the BI compilation and exposition platform along with Performance Point and SQL Server Reporting Services.  Now what we are doing will differ from a structured Data Mart because all of this data is coming from live systems and is completely dynamic in nature day to day.  Traditionally, Data Mart content is modified in purely a static data context (data modified as data).  But our implementation will be collating data from multiple systems internally and exposing in in a central BI Center.  I suppose you could call it a distributed Data Mart Model.

By leveraging a system like this, our executive team will be able to make decisions based on data that is dynamically gathered in real-time from all the internal business processes we leverage.  Decisions will be made more quickly, costs will be limited because we will be mitigating decisions made with false or outdated data, and our ability to react in the ever-changing world of consultant will greatly increase.

Stay tuned for Part 3 when I start talking about the 2 rules of Data Marts and how they apply in a real world scenario!

Data Marting and BI Development: Part 1

So recently I was asked by my superiors to head the development effort of a dedicated Business Intelligence Portal for our company.  This is to be built leveraging the existing SharePoint 2010 infrastructure we have in place for collaboration.  As I navigate the infant world of data marts, data warehouses, and enterprise class actionable data systems I will be giving my thoughts and musing on the process, the technologies, and the pain points so that hopefully someone out there won't be surprised with them.  Look out for part 2 and my introduction to data marting techniques and concepts!

Business Development is Marching Towards Mediocrity

In school and shortly after, I was a young idealist programmer.  Every piece of code I wrote was going to be ground breaking and every new algorithm I developed would revolutionize fields like signal processing, AI, and more.  But as I sure many of you have seen, the world of business/product development doesn't look anything like what we hoped it would be in all of our Comp Sci courses.  There we saw theoretical approaches, search optimization, concurrency, advanced data structures, etc.  All of these techniques have their place in the world.

Since I began my work as a SharePoint developer (dealing mainly with corporate clients), I've noticed an interesting trend.  Business software is marching towards mediocrity.  Now don't take what I'm about to say and take it to mean this is necessarily a bad thing.  Business software has a different focus than fields like Game Development, Technological Research, and Scientific Computing.  An enterprise needs a focus on non-technical requirements like up-time, migration-ready status, version/platform longevity, etc.  Though all of these elements exist for all developers, their emphasis during requirements gathering, design and implementation is extreme in the case of business class software.

I recently spoke to a colleague of mine about a company here in Atlanta, GA that is pushing the technological barrier in the enterprise space.  PinDrop Security is a software development house that offers products designed to detect and eliminate phone fraud, in particular call center phone fraud.  They are doing research and development in areas like digital voice fingerprinting, pattern recognition of speech, POTS and VOIP spoofing detection, etc.  This is exciting software.  This is what I went to school and studied languages and algorithms to develop.

I issue a call to all senior developers, development managers, and even CTOs in this world.  I ask you to take a chance and push some boundaries   We will never achieve more than what Microsoft and other platform vendors provide us without a careful eye on what could be possible.  We often see enterprise solutions restricted, restructured, or redesigned to allow for the complete use of the these products.  Don't let your license commitment to Microsoft dictate what it is that you build.  Think about your needs, what your users REALLY need to be more productive; think about what it is that your business needs form its systems.  Software is a tool and if we all use Microsoft's hammer, even when we may need something much more elegant, we've forgotten the most basic principles of this practice.  So provide support and encouragement to your teams in their innovations.  Allow them a little bit of slack to push an envelope that you are scared of because what they'll discover on the other side is nothing short of amazing and there is a very real chance that they will produce you something of true quality.

New projects for an old developer

I'm a developer.  I've been slinging code for several years now, mostly in the enterprise space.  I've designed, implemented, tested, and released several different large-scale enterprise products using .NET and the usual Microsoft stack including SharePoint.  I tell you this so that you'll understand what kind of work I do and I'm sure many of you are all too familiar with this same career or implementation process.

I've discovered recently that I am running out of challenges from a development perspective.  The business requirements that I see time and time again all contain complex business logic and validation, but the underlying implementation of such is relatively straight forward.  I need a more theoretical challenge.  Something that would require me to question my own assumptions about programming languages and to actually forge some kind of new territory.  My answer came in a realization from my past.  My Bachelor's degree was earned at Westwood college in Game Software Development.  This was my original dream, and for good reason.  I've decided to devote my personal development time to game development.  That is the purpose of this blog.  I will be detailing my dive back into that world along with all the pain points and all the triumphs.

As an exercise to refresh my knowledge and begin the process of reawakening my inner hardcore developer, I've decided to develop, from the ground up, a completely cross-platform game engine dubbed the Bad Monkey Game Engine.  The requirements of this are FAR to lengthy to enumerate here, however, I also plan to, in parallel, develop several small games with increasing complexity, using the engine as I go.  In this way, I will be forced to deal directly with every aspect of game development.  I've started a project over at Sourceforge.net to function both as the project distribution point as well as an offsite repository for my code.  Head over to https://sourceforge.net/projects/bmengine/ to see what has been done on the project so far.  I will be creating multiple sub-projects that will eventually become completely playable games built on top of the Bad Monkey Engine.