This is Preliminary documentation about UI Delegation Pattern to create Clean and Reusable Data Components and UI Markups in a Business Application.

UI Delegation Pattern

I would call it simply divide and rule, UI Delegation Pattern is narrowing the focus towards piece of UI Element on the screen and have a UI component to fulfill every specific business needs.

User Interface is the direct connection between User and your business application, thought MVC, MVVM models exist, but they are found to be very useful in CRUD business applications, but when your application is more then CRUD and it involves complex UI and media / hardware interfaces for scientific and entertainment related applications, MVC/MVVM are little difficult to maintain over huge applications. This pattern is already in existence at many places.

Initial Rough Draft

This is initial rough draft, I intend to collect problems and solutions and then I want to give a good name to this pattern, for now I am calling this as “UI Delegation Pattern” however I don’t know whether it suits the name correctly or not. But your opinions are most welcome, if such pattern is already named and exists then I am sorry if I am unaware of it, but please suggest if you know so.

UI Markup vs. Editor Files

Earlier Visual Basic or Visual Studio would create .frm and resource files that would contain the contents of UI design created in “Designer/Editor” inside an IDE. These Editors would create code that is required to execute at runtime to create desired user interface. However since Editors would create code with specific style and pattern, these editor files are more or less very structured and they are less buggy. Of course too much of code will certainly make Editor very slow but at the same time, the consistency of UI code is preserved. But you are too much dependent on the Editor and complex UI could be very difficult to work in Editors.

Where else UI Markup based on xml, gives full freedom to user to create highly complex UI, but at the same time there is no consistency left as everyone writes code very differently. Markups gets piled into one file and it becomes way to difficult to maintain and understand how it works.

Identify Markup Noise

My first approach is to identify problems with Markup Noise, so lets identify what is markup noise. When I see any html, aspx, php or such web scripts, the most annoying things I discover that in spite  great CSS and Modular programming support, the scripts becomes a big junk to maintain. Sure WPF / Silverlight are also text markups, hand coded markups becomes way to big junk to manage.

Bigger problem is to visualize markups as they become more complex. As more and more lines of code gets piled into markup, more time is spent understanding and troubleshooting the UI.

How to Reduce Markup Noise?

CSS brought a very useful way to reduce markup noise. Still lot of developers write inline styles and I hate to see that. Ideally nicely written code should have inline styles as small as possible.

Appearance Delegation

CSS comes under category of Appearance Delegation where your delegate the look and feel of your component to CSS file. In WPF and Silverlight the themes are styles are introduced and you must have all your styles and look and feel related code into resources only.

The markup containers (Connection Containers) such as Window / Page / User Control should not have any inbuilt styles at all.

Business Logic Delegation

This was not possible in plain html, but aspx, php etc did brought concept of server side controls where in you could inject business logic specific to components and reuse them. But these components (user controls / custom controls ) are only designed when they need to be reused, but even if it will be used only once, I would still suggest its better to have components as small as possible.

Example, Country Combo drop down , Gender Combo drop down, these boxes are nothing but set of options provided to user to select one of them. Now nobody makes specific custom controls for such items, but lets see a practical example.

Bad Markup Example

    <Window.Resources>
        <XmlDataProvider x:Key="CountryData" XPath="/Countries">
            <Countries xmlns="">
                <Country
                    CountryName="United States"
                    CountryCode="US"/>
                <Country
                    CountryName="United States"
                    CountryCode="US"/>
            </Countries>
        </XmlDataProvider>
    </Window.Resources>
    <StackPanel>

        <ComboBox
            ItemsSource="{Binding Source={StaticResource CountryData},
                XPath=Country}"
            DisplayMemberPath="@CountryName"
            SelectedValuePath="@CountryCode"
            />

        <ComboBox>
            <sys:String>Male</sys:String>
            <sys:String>Female</sys:String>
        </ComboBox>

    </StackPanel>

Now this file defines a resource xml for Country List, and it is bounded by ComboBox to list countries and another ComboBox defines inline items to be displayed.

Let us identify “Controls and Connection Containers”.

The window, that hosts Country List and Gender Combo Boxes is “Connection Container” in which we should only host controls and connect inter dependent properties of every controls.

The Connection Container should not have any code that is not related to either any other items or business logic of the container at all.

List of Countries and Gender items are not at all related to anyone else except for the drop downs. This is pure markup junk where we have so many unnecessary information.

Solution

Define custom controls as shown below and see how small our markup becomes. And move Country List xml to some resource file.

    public class CountryComboBox : ComboBox
    {

        public CountryComboBox()
        {
            this.DisplayMemberPath = "@CountryName";
            this.SelectedValuePath = "@CountryCode";

            Binding b = new Binding();
            b.XPath = "Country";
            b.Source = FindResource("CountryData");
            this.SetBinding(ItemsSourceProperty, b);
        }

    }

    public class GenderComboBox : ComboBox
    {
        public GenderComboBox()
        {
            this.ItemsSource =
                new string[] {
                    "Male",
                    "Female"
                };
        }
    }

The final Markup.

    <StackPanel>

        <local:CountryComboBox
            />

        <local:GenderComboBox
            />

    </StackPanel>

Wow, the markup is very clean now. Also the code is easy to understand.

Conclusion

The connection container (Window / Page / User Control) should only have interdependent declarations and should not have any items that are only specific to one control. It should be as clean as possible and it should execute all necessary business logic for the component.

Configuration items, even resource bindings should be avoided in Connection Containers. They should be narrowed down to custom controls as much as possible.

This increases readability, reusability and helps in troubleshooting while focusing by narrowing the target.

There is lot of buzz about LOB Business Form Layout for new platforms WPF & Silverlight, both capable of displaying rich content and extend functionalities of existing components so better then ever before.

XAML is better then earlier Rapid Form designers where you need to drag and drop within an IDE and the layout management becomes so complicated because no IDE is perfect. I remember upgrading from Visual Studio 2005 to 2008 and now 2010, most of our ASP.NET and WinForm projects would require lot of effort to work seamlessly in newer versions. The worst of all is, modifying layout without IDE becomes way to difficult.

In this presentation, I collected all important aspects of Business Form that we encounter in our day to day life and the complexity of forms make life terrible when clients request changes that takes hours on any IDE that sure breaks lot of code graphs and makes existing working system as completely useless code junk.

In UI Atoms, we at NeuroSpeech tried make set of components that are very easy to operate while going through tough change management.

SQL Assembly brings power of CLR (Common language runtime) inside SQL Database Stored Procedure which can overcome T-SQL programming pains of Intellisense, Debugging, Logging and Versioning.

Traditional Problems with T-SQL and Stored Procedures

I never liked programming in T-SQL, for various reasons listed below,

1. No Intellisense, of course it has been introduced SQL 2008 onwards, but still its far behind what we get in Visual Studio.
2. Source Code Control, although scripts can be added to source code control, usually its unawareness and unwillingness of Database Administrator / Designer to use it correctly and mess things up.
3. Less human readable, T-SQL is very difficult to understand if its not documented well.
4. Source code is open, it can be modified, so in short Database Administrator has complete access to alter business logic.
5. Very less reusable, unless you do copy paste, no way to standardize the development.
6. No support for advanced visual tools to design T-SQL Stored Procedures.
7. Deployment is difficult, minor alternation can cause problems.
8. Most of business logic involves, loops, if-else, switch statements, and it takes 10 times more code to execute such logic in T-SQL then any other programming language in world.
9. You need multiple language talent, one with either VB.NET/C# and T-SQL.
10. Difficult to Log information.

Introduction of SQL Assembly

Since SQL 2005, introduction of SQL Assembly has been of great importance, however not much of people still use it, because of unawareness. But SQL Assembly can be modularized and reused, and it exists as simple CLR assembly inside database.

And following are benefits of SQL Assembly, besides few listed in MSDN here.

1. LINQ in CLR Stored Procedure.
2. Better Source Code control over Visual C#/Visual Basic.NET project through Team System.
3. Intellisense.
4. Closed source, its safe.
5. Highly secure as Database Administrator can not alter business logic.
6. C# and VB.NET along with LINQ are more human readable, understanding and reusing code can be very easy.
7. Support for advanced documentation in comments.
8. Visual Studio Model designer can be used.
9. Reuse your Business Logic for other database vendor products, e.g. Oracle, MySQL (Explained below).
10. You can do advanced logging with log-4-net or custom logging.

Small Example of How to use LINQ in SQL Assembly

1. Create a simple Visual C# Class Library Project.
2. Add reference of “System.Data” Assembly.
3. Create new ADO.NET Entity Model, and you can create it from Database or empty model, the way you want to develop your project.
4. Add few entities, I am going to demonstrate entity “Cart” which is a saved shopping cart for the Customer.

Create a Stored Procedure Class

After creating model, lets create a class called “ShoppingCart” and add a static method as shown below.

public class ShoppingCart
{

    /// <summary>
    /// Creates new Cart if none exists and returns the Cart ID
    /// </summary>
    /// <param name="CustomerID"></param>
    /// <returns></returns>
    [SqlProcedure]
    public static SqlInt64 GetSavedCartID(SqlInt64 CustomerID)
    {

        // Set context connection=true
        ShoppingCartModelContainer context =
            new ShoppingCartModelContainer("context connection=true");

        // Query for exisitng cart
        Cart c = context.Carts.FirstOrDefault(
                        t => t.CustomerID == CustomerID.Value);

        if (c == null) { 

            // create new Cart
            c = Cart.CreateCart(0, CustomerID.Value, DateTime.Now);
            context.AddToCarts(c);
            context.SaveChanges();
        }

        // return the Cart ID
        return new SqlInt64(c.CartdID);

    }

}

This code demonstrates how easy it is to write SQL Assembly, now here is the ultimate benefit of everything.

Ultimate Benefit

You can use your Business Logic in multiple vendor databases, like for example for Oracle and MySQL, you can reuse your SQL Assembly and just change your connection string. You can reuse your business logic in your front end code with simple wrapper.

And with SQL Server, you can reuse this assembly inside SQL Database.

Looking at the innovation driving inside Silverlight, it is getting more attention then the former WPF. However having very similar syntax and .NET as base platform, both WPF and Silverlight are very different in approach, design and use. Today lot of developers live in dilemma of WPF vs Silverlight and it has now become a great trade off as similar to speed vs size.

Lets compare the need and features of both and make the trade off very easy to understand.

History of Microsoft Development Platform

We all loved Visual Basic, what did it provide? a rapid development platform to make quick business applications. By business applications I mean, the one to track your purchases, your activities which are just probably relational database and reports. Business Application makers didn’t have to worry about device drivers, system services, hardware and component versions.

Visual Basic provided very thin layer of application development for rapid turnaround time of small scale but huge in demand business applications. These applications require very less programming language expertise but more of business expertise. But it had limitations, you could not create a huge resource intensive application like CAD programs, Media editing programs, servers, games etc.

On other hand we had Visual C++, a huge development framework with possibility to do almost everything that was possible on Windows Operating System. Where you could fulfill need of any program, be it CAD, games, media editing, or servers etc.

But the development time required was very high, performance was great and the intelligence required to create system applications was very high. Getting talent to build such applications weren’t easy and even it is not easy today, because you need real in depth knowledge of how every part of the system functions. There is more dependency upon device drivers, component versions and there was no drag and drop.

Apart from any other platforms existed so far, Microsoft clearly and distinctly divided development into two major categories. Rapid vs System, or Native vs Thin applications.

Evolution of .NET

Ever since c# came into existence, we started seeing quick shift from tedious MFC/COM/ATL to c#, because all technologies were simply extension of c/c++ and each needed separate runtime to exist. Even still today, some of my components are written in ATL/COM and compiling them in newer version of Visual Studio is always pain. Compilation time is huge.

Where else, .NET based project took less time to compile, even the 1.0 versions are still in use and still in good shape.

Earlier it was expected that it will only replace the Visual Basic segment of programming, but somehow all devs started liking it and preferred it as an alternative to ATL/COM/MFC as its native capability and speed provided easy development and quick turnaround.

Windows Forms did provide an alternative to Visual Basic. But there was lack of more powerful, GUI interface in .NET. So came WPF. Not only this, .NET being neutral, the compiled apps are independent of 32 bit or 64 bit platforms.

Windows Presentation Framework

As we clearly see, Microsoft has its plan to make WPF as its primary GUI framework against all Win32 components. WPF is very easy to write as it is very simple XML, XML is very easy to read, and its very easy to organize, even the editors and compilers need less overhead in parsing and taking decisions based on XML.

The challenge here was to imitate almost everything that existed in Win32 inside WPF, and WPF became very feature rich, but once again it requires more intelligence to write complicated and big applications. Having talent already expert in HTML+JS area, and transforming them into WPF talent was little difficult.

Plus the way WPF was built, it was built more tight coupled with system operating system, just like the way MFC/ATL/COM were built. But WPF is very fast and very rich in features. It is now possible to write CAD, games and such high performance applications in WPF which can utilize latest hardware technologies.

And it is exact alternative for COM/ATL/MFC today.

Silverlight

Realizing that converting HTML+JS and Visual Basic devs to WPF was more challenging, silverlight pop in to existence, Which is very light weight, a smaller counterpart of WPF, although the syntax remains same, the ability to develop apps are limited in silverlight as it is not very tightly coupled with operating system.

However, you can develop apps pretty fast, but only focused on business logic, which here is one step ahead amongst all being platform independent. The devs are not required to have in depth knowledge of system, nor silverlight are more dependent on hardware, resources or other components.

But you cant develop high performance apps like CAD, games and media editing apps.

Indeed, it is an exact alternative for Visual Basic.

Conclusion

You need to choose WPF when you are creating any application of following type,

1. Interfaces with custom hardware.
2. You need to control memory and control the performance of app.
3. Create application that requires very high memory, CAD programs etc.
4. Connect to system components and used to manage system, hardware.
5. Want to connect to native system via COM/DLL interfaces.
6. Media (Audio/Video/Picture) editing applications.
7. Where your target audience can download and install .NET platform easily.

However you can use Silverlight for following alternative type of applications,

1. Provides smooth and quick animation on the web.
2. Make Rapid Business Applications, which can connect to your enterprise apps.
3. Make typical CRUD applications.
4. Registration, Dashboard type of applications.
5. Lightweight gaming and interaction websites.
6. Run in browser, with very small installation needed.

Coming Up:

UI Atoms from NeuroSpeech, a set of controls to develop rapid business applications on both WPF as well as Silverlight. A true alternative for DataForm on Silverlight and WPF, to have unified coding for Business Apps. Please click here to check the preview.

Apple announces iPad, a bigger version of iPod Touch. With a big screen of 1024×786 pixels. And it comes with basic need of surfing internet and be online. So called magical and revolutionary device does not support Flash and Silverlight yet. And most of business apps today, built upon RIA are running either on Flash or on Silverlight. And IT Devs are heavily investing in RIA technologies due to snail speed of WWW group. However, the device can replace a laptop for most of users who just spend time on internet doing their regular business, studies or entertainment. So lets analyze what will be future of Web for different type of users.

Students

Today students need a good web browser and an internet with lot of social media websites to study, interact, entertain and play games. Apple iPad is a best bet for students. Do they really care that it does not support Flash and Silverlight yet? Answer is no. They will certainly buy iPad, come to the website, and if it has Flash or Silverlight, they will call website’s support instead of Apple’s support and say,

Your website is not working on my iPad.

Sales Reps and Business Owners

Looking at the cost, and attractive looks, Sales Reps and Business Owners will buy iPad because they usually do not type anything great whole day, all they do is, look at websites, present demos (which are powerpoint or keynote), and access their office through online business portals. For all these purpose they will be very happy with iPad. However, once they come to your business app through online portal, they cant see Flash or Silverlight, they are not going to call apple support. They will call IT devs and say,

What the hell IT devs are doing, cant they make our business app work on iPad?

Independent Professionals

Independent professionals like Doctors, Lawyers, Photographers, Actors .. anyone who probably does not sit in front of PC for more then 2 hours a day, will certainly like to get rid of big laptop bag with 4-10 external wires and device connectors. All what they do mostly is, check their emails, surf the websites, fill simple forms on some business web apps and which they can do it on iPad. Today iPad may not have great drawing tools but some professional apps on App store will certainly provide capability of tagging, drawing and marking tools needed by these professionals. Today lot of business applications are on Flash and new ones are coming on silverlight. But iPad will not support !! , once again, they will pickup the phone to vendor and say,

Your website is not working on my iPad.

Who needs any of following?

Clerks,  Graphics/Media artist and IT Developers, which comprises of small part of computer users. Because their applications and workspace requires bigger screens and multiple devices, connectivity and computing performance. Apple’s iPad will be useful only to test the business apps and for other entertainment to this group.

It seems iPad will spread in Home Entertainment, Education and Independent Professional sectors very fast because of low cost solution. And that will shake the developer community.

Troubles for IT Devs

1. Current investment in Flex or Silverlight is at stake, as usage of mobile devices is increasing heavily, supporting business apps on these devices will not be possible until device OS creators, Adobe or Microsoft work towards supporting RIA technologies.
2. Silverlight/Flex are heavy and costly to run on battery based devices, they will consume more cpu and battery to create consumption problems.
3. Rewriting apps in light HTML5 is a big challenge, training, planning and supporting all existing modules will be a big pain. However looks like World Wide Web Consortium (W3C) has been sleeping for 10 years to create features of RIA required today. Need of consumers were high, that created innovations like DHTML, Flash and Silverlight, and the problem was, W3C never observed in market what consumers (Internet users) really need.
4. Even if you decide to invest heavily on iMac and decide to make apps that can run on iPad, your Apps are at mercy of Apple Employee doing approval, here is the funny process.

Death of Free and Open Internet

IT Devs will slowly loose the independence over distributing apps against open free internet distribution. Today any release, distribution of media (song, movie, play) etc are controlled by major Labels.

If The  Intelligence will be controlled by major Labels in IT with narrow route to success by Apple, Google, Microsoft etc giants, then the growth we saw in last decade in IT industry will slow down very rapidly and independent innovators will become slaves to major labels.

Will Bubble of RIA Burst now?

Once again, just like Dot Com, if RIA (Flex/Silverlight) Bubble bursts and all IT Devs will be blamed again.

Its less of a risk then getting yourself a liposuction or a nasty boob job ..

Before I answer WHY let me answer WHAT! For all of you who don’t know what I am talking about watch this video “The Photoshop Effect”

Why Retouching

On a serious note, Human psyche has been on a constant Endeavour to achieve perfection. Retouching is a means to persue that human pursuit. How else would you preserve and portray the invaluable momentary short lived tequila shots of your life. A retouched image would give you that well deserved kick time and again.

Besides not all of us can carry a Cindy Crawford mole as a beauty spot the way she does. Come on, would you want that dark spot seen in your wedding snaps. You would rather get them retouched. Similarly you would more often then not, want to get a brighter retouch-er if you are from the Naomi Campbell or a Hale Berry progeny. "I don’t mean to be a racist here" unless you want to spend a bomb to look as good.

Retouching the History

Beginning from our inception, Adam was perfect as a first masterpiece, as a human being, but as a man he was lacking. Later, Adam with Eve became a perfect couple until they fell prey to Satan’s handiwork and since then we’ve been living in an imperfect world. I need not elaborate on this any further and no wonder everything around us is (believe it or not) imperfect.

Retouched – Do We Even Know?

BTW If you see a problem here, sure do something, lets run a campaign to make it a mandate to put an * (asterisk) symbol on every retouched image. I bet you would see 99.9 % of photographs retouched in any magazine or billboard or for that matter any print media with an asterisk sign, the way we notice "*" or "Conditions Apply" on every shop running a SALE.

Lately there has been a lot of hype around retouching and its adverse effect or so they call it, just because some nuts went on an indefinite dieting spree to fulfill there fantasies of attaining that perfection portrayed by a retouched image of a model. I have a question here, if some one ignores that ‘asterisk’ on a FOR SALE sign board and happens to pick up stuff and queued up at the counter only to realize that there was some other exception, what do you think would happen. Wouldn’t you ask her to go back to shopping basics. She’s not duped! An asterisk is supposed to be taken for granted.

Retouching  & The Craving Need of It

As I proclaimed above I see no moral hang-up in retouching a photo but the reality is that it can change a picture from something you dislike to something enjoyable. We all understand the difference between tweaking the truth and portraying the expected. Perfect moments do not always prove to be perfect and that’s where we need the so called digital liposuction.

Let me sum up with a practical example:

A stunning model sitting right in front of you that has potential but several flaws. By several I mean the following two major flaws apart from a few here and there that can otherwise be taken care of:

Dull Hair
has a pimple right on the face of the camera lights

What do you do ?

1) Hand over the best shampoo available and wait for the results + wait for the pimple to even out.
2) Click the snap and retouch it. Its simple and worth while.

I leave it for you guys to decide. Any takers!!

For all of you guys who want to try the first option you need to keep visiting my blogs for the rest I know you guys understand how difficult it is to get a boob job done, and how difficult it is to get that perfect look unless we go to RetouchMAX.com

After visiting Microsoft TechNet Seminar everyone was quite excited about Windows 7 and everyone was saying that Windows 7 is far better then Vista and its now ready to go alive. I did stick to my plan of upgrading my development machine to Windows 7 only after Service Pack 1 arrives. But everybody argued that no its ready for developers even. Fine I tried to install it on my laptop which I rarely use for development and here is the experience.

It took drivers and it was pretty faster then Vista as far as installation goes, but then immediately I found two problems.

Move Files Progress Bar Bug

I took backup on my other D drive and I have both administrator and basic user and while copying files back on basic user I found this bug. When you are trying to move files, and if you are an administrator and it doesnt need any UAC permissions then you can see progress bar as shown below.

But if your destination folder/drive requires some UAC permission then the UAC dialog box comes and when you accept everything after that the progress bar no longer displays progress as shown below.

This happens if you try to copy something into windows/program files or anywhere for some other users when running from basic user.

And we have to wait endlessly for this window to get closed automatically and that probably proves that operation was successful.

I am really disappointed to see bugs on very critical operations like file operations are most crucial. Even on Vista, Move operation did not work correctly as it did not delete the source files/folders and that leaves us with confusion.

Canceling Move Operation Creates Inconsistencies

While the Move operation is on, if you decide to cancel, you are left in bad position because then you are left with half files from old location and half in new. Sure its not great problem as we can consolidate immediately.

Cant Install Remote Server Administration Tools (RSAT)

This is really great bug, because on development machine, we really cant live without RSAT. We need them to manage servers. I found this link from live search, RSAT for Windows 7 . When I try to install this update it gave this error message “This update is not applicable for your computer”. Further more in Event Viewer, I got this message.

Windows update  could not be installed because of error 2149842967

I did report this bug to lot of places, hope it gets resolved soon, but I think on Microsoft Download website they should put some feedback about the item being downloaded.

Visual Studio 2008 Installation Problems

I am not sure whether VS 2008 installed successfully or not, because although it said it was successful, there were certain errors in Event Viewer.

Product: Microsoft Visual Studio 2008 Professional Edition – ENU — Error 1935.An error occurred during the installation of assembly ‘Microsoft.Windows.Design,
version=”3.5.0.0″,
publicKeyToken=”b03f5f7f11d50a3a”,
processorArchitecture=”MSIL”,
fileVersion=”9.0.30729.1″,
culture=”neutral”‘.
Please refer to Help and Support for more information. HRESULT: 0×80070005. assembly interface: IAssemblyCacheItem, function: Commit, component: {7A515F16-7D3F-4A41-910A-E704ABC9E8CA}

I havent yet started using VS on 7 but I will soon and I am sure there will be many problems.

Cant Install Visual Studio 2008 Service Pack 1

However when I tried to install Visual Studio 2008 Serivce Pack 1, it didnt install and gave following errors in log.

Product: Microsoft Visual Studio 2008 Professional Edition – ENU – Update ‘KB945140′ could not be installed. Error code 1603.

And another error at the last,

Product: Microsoft Visual Studio 2008 Professional Edition – ENU – Update ‘Visual C++ 2008 SP1 Design-Time Components for x86 – KB947888′ could not be installed. Error code 1603. Additional information is available in the log file C:\Users\***\AppData\Local\Temp\Microsoft Visual Studio 2008 SP1_20090913_140916022-Microsoft Visual Studio 2008 Professional Edition – ENU-MSP0.txt.

Cant Install SQL 2008 Management Tools

I tried to install SQL 2008 Management Tools, it gave me compatibility issue and warned me that you must install SQL 2008 SP 1, I agreed. And I went ahead with installation and Installation stopped because Visual Studio 2008 SP 1 was not installed.

Conclusion

I think I am on conclusion that Windows 7 is not yet ready for Developers and they should wait for Windows 7 SP1 to arrive.  And it should only be used by Testing Teams to test the products.

The quest to store large number of files in database or on regular file system began at a time when number of files to manage went beyond 1 million in one of our project. We started noticing a huge drop in performance of our windows server. After investigating we found out that using pure file system to store more then million files is really a bad solution. And following are key points.

1. Each operation of file open/close requires kernel operation, because it needs to do various security checks.
2. Every file operation requires some minimum locking mechanism which uses your cpu in kernel mode.
3. Above operations being kernel operations, they slow down entire server even if your file access may be of least priority then any transactional database operation.
4. Each services on server slows down.
5. Even operating system itself manages a file system database (typically known as File Allocation Table and similar names in other systems).
6. These databases are optimized but not for millions or trillions files. They are good to work under million files.
7. It causes huge fragmentation on your storage media.
8. Not only media, but even File System Database gets fragmented, leading to more slow down.
9. Durability of storage media also can reduce because of too much fragmented usage.
10. Backup and Restore operations take too long time. You can easily conduct an experiment, on any ftp client, try to upload only one 1Meg file and then try to upload 100 10KB files, although the total transfer is only 1Meg, you will notice uploading 100 files of smaller size takes longer.
11. Keeping track of changes/deleted items are even more difficult.
12. Tools for replication and change tracking for file systems aren’t available for huge file storages.
13. The web server will need write access to some folders which is always a big security hole.
14. And file system crash = operating system crash and thats  a huge challenge to bring it back up.
15. For transferring files you need to open ftp and develop some tools to replicate them.

Finally, after all these observations, we started considering other options.

1. Network File System, it looks promising, but even at other end of node, its simply an operating system. It still has same problems as mentioned above.
2. Distributed File Systems like Hadoop and other tools require too much of learning, management and I think they are good when you need file storage as huge as Facebook, Yahoo etc. We are mid size company and our need is 1TB for now.

Database Solution:

Finally we began testing database solution with MySQL and then we moved onto MS SQL, however both have good advantages and disadvantages that we will discuss further.

We have following advantages over file systems, and also I have provided solution detail later on this article.

1. Database systems have matured enough to handle large data easily today.
2. They use small number of files to store all relational data in well organized form.
3. They keep their database files open all the time, so the kernel operations of locks are reduced.
4. You can consider as if your load of entire server is now concentrated only on your database.
5. Databases are much advanced in terms of their scheduling of queries, you can reduce priority of your file database and database will easily offer more performance on other transactional databases.
6. Data storage is transactional, where else file systems are not completely transactional. And file systems require more programming to make them transactional.
7. Remember if its your customers data, even if its a small picture/icon any video.. if they are paying for your service then its better to store those media files in transactional storage.
8. Database crash doesnt crash entire server and nor even other databases on same server.
9. Backing up and restoring databases are easier because of the inbuilt tools.
10. Replication on database are much easier.
11. Change tracking is available on database.
12. Database access is standard its not dependent on ftp/http like tools.
13. Database can easily take over failures of neighbor nodes in network.
14. You can easily query database and do multiple operations on files.
15. The biggest point is, your web server does not need any write permission on any folder !!

Comparison between MySQL and MSSQL for file storage

1. MySQL offers replication over internet, MSSQL doesnt, you will have to write your own replication program.
2. MSSQL offers auto grow option for database file, with which you can reduce fragmentation. MySQL’s innodb doesnt have such option for database file auto grow method which leads to more fragmentation.
3. MSSQL offers easy incremental backup/restore and more over, in case of crash, you can use actual data folder files to restore, where else MySQL’s innodb doesnt support hot backups so either you buy their hotbackup tool or rely on mysql’s backup tool which is not incremental.
4. MSSQL costs $1600 for standard edition, where else MySQL innodb is free, but hotbackup or incremental backup tools will cost exactly the same. MSSQL wins over mysql for offering better file management and increasing disc durability.

Sample Solution

After reading various articles, we came to conclusion that storing entire file in one blob or image doesnt make sense as it will deteriorate performance like anything.

So we came with easy solution, which is already used by existing file systems. Thats “Breaking down file into smaller blobs, max 512kb”.

As MySQL and MSSQL both have network packet size limit, you should not store more then 512kb in one row. Otherwise you will have to increase network packet size and that will slow down database server. Following is easy small table structure we used.

Table: NSFiles

NSFileID (Auto Number) (bigint)
NSFileName (varchar 50)
NSContentType (varchar 100) ** This is most important for web access

Table: NSFileBlobs

NSFileBlobID (Auto Number) (bigint)
NSFileID (bigint) (indexed) (Reference of NSFiles.NSFileID)
NSBlobIndex (int)
NSBlobData (medium blob or image)

Following are Storage/Retrieval methods specified in C# with using Linq or any similar ORML tool.

//Upload Handler
NSFile file = new NSFile();
file.NSFileName = .....
file.NSContentType = Request.Files[0].ContentType;
file.Insert(); // <-- insert into database and get new unique NSFileID
byte[] buf = new byte[512000];
int i = 0;
using(MemoryStream ms = new MemoryStream(Request.Files[0].FileData))
{
int count = ms.Read(buf,0,512000);
while(count>0)
{
NSFileBlob blob  = new NSFileBlob();
blob.NSFileID = file.NSFileID;
blob.NSBlobIndex = i++;
if(count==512000)
{
blob.NSBlobData = buf;
}
else
{  // last blob will always be less then 512kb
blob.NSBlobData = new byte[count];
Buffer.BlockCopy(buf, 0 , blob.NSBlobData, 0 , count);
}
blob.Insert();
}
}

And following is simple code for Download Handler

// Download Handler
long fileID = long.Parse(Request.QueryString["FileID"]);
NSFile file = NSFile.Adapter.Get(fileID);
Response.ContentType = file.NSContentType; // important for web browser
// following line causes "Save As" box on client
Response.AddHeader("content-disposition","attachment;filename=\"" + file.NSFileName + "\"");
// following loops over all file blobs whose NSFileID is fileID and they are sorted by blob index
NSFileBlob.Adapter.ForEach(
NSFileBlob.Schema.NSFileID == fileID,
NSFileBlob.Schema.NSBlobIndex.SortAs(),
(t)=>{
Response.OutputStream.Write(t.NSBlobData,0,t.NSBlobData.Length);
}
);

Conventional AM/PM time format is very confusing. At NeuroSpeech we decided to use new format to distinguish visually the day and night format. Rules are as follow,

1. Day time is considered between morning 6 to evening 5.59 as assuming range of average sun shined day.

2. Night time is considered as 6 of evening till 5.59 of next day morning.

Representations:

Print and computer

White background:

1. Day time should be written with black box around the time and white background inside the box with black text color.

2. Night time should be written as black background of text with white color text. No border.

Black Background

1. Day time should be written as white background with black as text color with no background. No border.

2. Night time should be written as white border around the white color text.

Handwritten Format:

1. Day time should be written as normal dark ink color.

2. Night time should be written as normal dark ink color with a border around it.