Wow, just look at the “roll call” from last night's geek dinner in San Francisco. Thats fourty-nine people. Well, sure they had some well-known names there, but then again we also have some well-known names at our own Atlanta events (at least as far as I am concerned).

I am quite envious. We only had what, five people at this month's event? We can do better. Is it just a matter of “getting the word out”? I know it cannot be because of the holidays... the SF dinner was held between xmas and new years. Southerners are noted for their sociability and accomodation - such a recurring event should fit right in with culture here.

I have been battling a pretty strange stability problem with a particular .NET application for about a month now. The end-user(s) first reported it around the thanksgiving holiday... but my gut feeling (and previous experience) tells me that it really surfaced earlier, and they just waited a while to inform me about it.

In a nutshell, the application just “disappears” randomly. No trappable exceptions. No warning signs. It just vanishes as if the main form had been closed (it is a Windows Forms app). There is no “reproducible” pattern of actions that lead up to a crash, sometimes the user can work for hours before getting a crash (from a seemingly benign activity), other times it crashes every ten minutes.

My first suspicion was a threading problem. I know this is a classic symptom of a background thread trampling the UI thread's GUI objects. However, I removed the (very little) threaded code from the application, and the problem persists. There are now absolutely no references to the thread pool, nothing attempts to create a manual thread, and nothing makes use of BeginInvoke/EndInvoke with delegates or web service proxies.

My second suspicion was that perhaps my cache refreshing mechanism was to blame. At application startup, immediately after login, the client program downloads “lookup tables” from the server. These are things like basic customer/brand/etc information that is often referenced, but for performance I choose to cache these lists locally. They are just simply classes (not datatables/datarows) to keep the overhead as low as possible. On a regular basis, there is a timer (was originally a System.Threading.Timer, but replaced with a Windows.Forms.Timer when I removed all possible multi-threading) that fires, and any new/modified cached items are retrieved from the server and merged back into the local cache (uses an audit time-stamp to determine what to pull down, if anything). I didn't think this mechanism was to blame, but just in case, I have completely disabled the cache refresh timer. The crashing bug still persists.

After that did not work, I started looking into “nonstandard” things that I might have been doing in my code for clues to this elusive bug. The biggest thing was that I was using CallContext.SetHeaders() to pass along security principal/identity information as out-of-band data along with every Remoting call back to my remote app server. This was used to enforce call-level role based security. I gutted this code from my application on both client and server ends. The bug persisted.

Now I am at the point where I begin to grab at straws.

I created a new TextWriterTraceListener during application startup. This gets pointed to a unique filename every time the application starts up. I basically went through nearly every method of every class/form in my client codebase, and added tracing information that gets flushed through this new TraceListener. We are talking LOTS of tracing information (I ended up going back later and commenting some of it... the trace files were just getting too big to work with). This tracing data showed that sure enough, the application was just terminating hard on an unpredictable basis. One thing I did notice however, is that 1 in 3 crashes were during a Remoting call return... as the response was being deserialized.

Hmmm. I cranked up CLR Profiler to gain some insight. I also used Windbg and Vadump as described in this article to peek into what was really going on in my application. It seems to me that the BinaryFormatter used in Remoting serialization is allocating some significantly large byte[] structures (really big... like tens of megabytes). This shouldn't be a problem really, given its only temporary allocation. However, what was also happening is that these large allocations were surviving into Generation 1 and 2 of the garbage collector... meaning they were not getting cleaned up until a Gen-2 Collection was triggered.

My hunch was that a Gen-2 GC Collection would become inevitable eventually... and this collection was in turn somehow causing my crashes. To test this, I decided to begin applying a call to the following method liberally throughout my application... mostly in places that I know were high-traffic (such as just prior to Remoting calls):

  public static void GCCollect()
  {
   CrashTracing.Trace(4, "GC", "Forcing a gen-2 GC.Collect() operation");
   GC.Collect(2); // force a FULL collection
   CrashTracing.Trace(5, "GC", "GC.Collect() finished, now waiting for pending finalizers");
   GC.WaitForPendingFinalizers();
   CrashTracing.Trace(5, "GC", "GC.WaitForPendingFinalizers() finished");
  }

The idea is that if the garbage collection was causing the crash, this would surely draw it out faster. And sure enough - three out of four crashes since this change occurred on the line GC.Collect(2);. All the other crashes were during a period of activity where it could be reasonably assumed that an automatic GC collection was triggered (loops with data allocation, etc).

So - now here I am. I am pretty darn sure I have a problem that is causing the garbage collector to crash on either the collection process, or I suppose possibly during the finalization process (since that runs on a background thread, and begins running before Collect() returns). My problem now is that I am at somewhat of a dead end with debugging this. There is no way to step through the GC processes, and there is no event I can attach to that either traps GC errors or that notifies when an automatic Collect is triggered. My best guess is that either a .Dispose() or a finalizer is raising an exception, which is bringing down the whole CLR. But I can't figure out a way to identify the culprit object... which may very well be out of my control (could be a 3^rd party or CLR class). The only 3^rd party controls being used are a few NetAdvantage UI controls, and the Acrobat Viewer activeX control (the interop assemblies are often not even loaded into memory yet when it crashes - so very doubtful my problem is related to this one).

Has anyone encountered a similar problem, and lived to tell about it?? Has anyone read a post or article by someone else that may have tackled this beast?

Michael lays out a position of why DRM is not evil. To be fair, he makes the disclaimer that these aren't necessarily his views, but he is just presenting an extreme view to drum up discussion. So here is my extreme view on the subject (since I didn't get to participate in Michael and Paul's discussion):

DRM is evil.

It was NOT the will of Bon Jovi that decided the music my wife downloaded from Napster last night should ONLY be played from the EXACT directory it was downloaded to (I got her a iPod clone and a gift card to Napster to get her used to music downloading). It wasn't the actors, director, or camera crew that wanted to force-feed 10 minutes of freaking commercials and trailers in the "unskippable" track of the Riddick DVD I watched the night before last. These things were done by the DISTRIBUTORS of the media. And thats where the problem lies. The distributors hold all the cards, they control all the (realistic) distribution channels. If you want exposure as an artist, you must go through a distributor. Just publishing your own material through the internet is not a substitute for this.

The “just don't buy it if its too restrictive” is not a valid argument. For it to be valid, you would have to be offered a choice. There is not choice here. There is no alternate (legal) channel for me to obtain a Chronicles of Riddick DVD that does not include ten minutes of unskippable commercials.

Not to mention DRM as currently implemented is blatantly illegal according to current copyright law. Why?

1. The restrictions placed by all current DRM schemes never expires. Copyright law expressly states that after a period of time, all protected work becomes part of the Public Domain. You cannot buy a DRM-protected media format today that will become “unprotected” and usable in 15 years.
2. As an author or artist or publisher, you do NOT have the right to dictate HOW or WHERE media is consumed once you sell it to a consumer... no matter how much you would like to have that control. You only have the right to Sell or to Not Sell it, and to Prevent Unauthorized Redistribution. DRM currently does not really address Unauthorized Redistribution, all it does is hinder the ability of the consumer to use the media as they see fit.
3. “fair use” law states that the consumer of published media has the express right to “first sale”. In other words, you can buy a music Album from the original distribution channel and sell it to someone else later... as long as you do not retain any copies of it yourself. Think of “garage sale” here. DRM restrictions do not allow for this.
4. “fair use“ also says that the consumer should be allowed to consume the content in private however they wish. Napster preventing my wife from playing a song from her MP3 player, DVD encoding that prevents me from playing a ripped movie on another machine in my home, a PDF that is marked to disable text-to-speech reading,... all of these things fly in the face of “fair use“.

Another problem I have with DRM is the fact that it is really just a tool to prop up market prices. It doesn't really address intellectual/artistic copyrights, but what it does do is prevent individual redistribution. Therefore, the exact same movie can be offered in two countries/markets, for wildly different prices at different introduction dates. A consumer in one market cannot resell (under “first use”) the content to another consumer - simply because of the region locking mechanisms that are a part of DRM. This is definitely not fair market-based capitalism at work.

Posting this entry as more of a “bookmark” reminder to myself than anything... but this information is downright useful.

I have also added Rico Mariani's blog to my reader (the person who authored the above article). Now to go have fun dissecting some troublesome (i.e. “unstable”) code which appears to be going belly-up during GC collections....

It's almost the beginning of a new year, and I know many corporate budgets are about to get replenished. Projects will be kicking off, and employers will be hiring.

It's also a coincidence that my current contract work is coming to an end. Well, sort of. It's winding down enough to where I can only really call it “part-time”, and therefore I need a more significant source of [steady] work soon.

I am posting this on my blog, as I really do not like going through the job/contract boards, and would prefer to not have to place my resume on them. Besides, “word of mouth” references are almost always better for everyone.

So if you, your employer, or an employer you are familiar with is in need of a solid .NET developer/architect/dba who can hit the ground running and exceed expectations, then please get in touch with me (keith@mindfusioncorp.com). I am always open to contract as well as “w-2” employment, although most of my work in the past six years has been contract consulting. Something in North Atlanta (Cobb or Fulton) would work out best, and yes I am open to infrequent travel. I would even consider relocation if the work sounded really cool.

I am out of town visiting relatives for the holidays, and will not be back until the 27^th of December. So if I do not respond quickly, it's because I am stuck in the middle of nowhere (a farm in the center of South Dakota) and do not have access to my email :) I will get back to you once I return to civilization though!

I have been putting it off for a while now, but today I went and took my final certification test needed for MCSD (70-300), and passed. So now thats MCAD, MCDBA, and MCSD. I suppose if I go take one more test I could also get MCSA, but I am not sure if that would be of any benefit given my profession (architect/developer).

Thanks to Eric Engler for sending me the discount voucher number, which saved me 25% of the testing fee. He also took the test today, and passed! So congratulations to Eric as well!

I sorta forgot about this until Shawn posted a reminder on his own blog. The next Atlanta Geek / Nerd Dinner is tomorrow night (dec 16). Same place as the last one (Al Azteca on Roswell Rd). See everyone there!

Kirk noticed my WSE3 wishlist, and points out that some of the things I ask for actually have already been included in .NET Framework 2.0. Cool beans. I was not aware of these new features before, given that my exposure to VS2005 has been extremely limited so far (not enough hours in the day). I also like that new /serverinterface command-line switch for wsdl.exe... very handy.

So scratch off a few of my requests!

However, I still hope to find serializable SoapExceptions, WSE3 on Compact Framework, message payload schema validation, and automatic resource searching for schemas.

With the new “extendable” wsdl.exe, I hope that the WSE folks can package some prebuilt example SchemaImporterExtension assemblies that are capable of generating SoapClient/SoapServer stubs from an existing WSDL contract. That would go a LONG way towards assisting contract-first development.

Kinda cool. If anyone in Atlanta is playing with one of these, I would love to see some functional demo(s) at our monthly .NET Mobility User Group meetings (yes, I realize the forums are down a lot... just keep trying).

It's a bit too far out of my realm to dig into personally, but I still would like to see what others are doing with this platform. The applications seem limitless really.

Unless you are living in a cave, you probably already know that Robert Scoble and Shel Israel are doing a book about the impact of blogging in the world of business. This book is currently title “The Red Couch”, after the couch in Scoble's living room where he and Israel hatched this book idea.

I think this is a really cool idea, and I expect to be following along with every step of the book's creation. You see - they aren't just writing a book about blogging... they are writing a book by using blogging. They hope to keep as much in the open as possible, relying on reader comments for a portion of the content. In some ways, it will have more “authors” than any other literature publication. The blog for the book's collaboration is currently being hosted at MSN Spaces, but they might be moving it to a private hosting site in order to enable a few more customizations.

I suppose one could call this “Open-Source Authoring“.

Kirk points out that there is a wish list being collected for WSE3. I had almost forgot about this.

Some things Kirk mentions, which I agree would be a HUGE boon to WSE3:

• Better documentation (just try finding a complete listing of the valid transport strings in WSE2...)
• Whidbey support
• Integrated UI for contract-first development (OMG this would be sweet!)
• SoapService templates in the IDE (similar to current ASMX templates)

But here are some other things I also thing are really worthwhile:

• WSDL generation support in SoapService (I realize this is probably too hard, but I have faith in the WSE team's mo-jo)
• wsdl.exe or similar to support creation of SoapClient proxies!
• Might as well give us an extensible xsd.exe compiler that actually allows us to control the xsd->class mappings (properties versus fields, COLLECTIONS versus arrays, etc). I say this because I doubt the asmx team is going to do this for us.
• Make SoapException support ISerialize! You may ask “why on earth would that be useful?”... well in one specific example, I have a business object that a client application interacts with via Remoting. The business object itself is a proxy to a soap service (Sql Reporting Services to be exact). Guess what happens when the service proxy fails and I dont wrap the SoapException.Message into a generic Exception? Yeah, serializable SoapException would be nice. *maybe this one is not really relevant to WSE team, its more a part of the core framework I think
• WSE3 ON COMPACT FRAMEWORK!!! Most importantly, WS-Security and WS-SecureConversation! I don't care if you have to roll the entire System.Security.Cryptography from OpenCF.org into WSE3 to make this happen. SOA that incorporates mobility devices is just plain GIMPED without this.
• An attribute valid on SoapClient/SoapServer methods marked with [SoapMethod()] to enable xsd schema validation.
• Another cool thing would be if the schema cache locator stuff would automatically search embedded resources while validating a schema. Right now, this must be coded by hand (which really isn't hard, it's just something that would be “handy“).

That's about all I can think of for now..

I had already seen a small demo of this at Best Buy recently. I wasn't convinced to buy one by what I saw. But I was certainly still greatly interested.

Then Michael Earls posted about the Digital Joy exhibit over at Perimeter Mall... and I just had to go check it out (again). The folks operating the exhibit really knew how to show it off. I also learned that for how I want to use it, I don't need the more expensive z545... the slightly cheaper z540 will do just fine. Either way, I also would need to buy an x5400 Media Center Extender for each bedroom where I have a TV. I am not even considering the “desktop” variants from HP, nor the offerings from any other manufacturer. I want this to “seem” like a “VCR” type device to visitors in my home (the simpler, the better).

To make a long story short... now I WANT one of these. I want to download movies instead of renting DVDs. I want to download music from iTunes and AllOfMp3.com. I want to pipe streaming internet radio through my stereo. I want my wife to be able to show our wedding video (currently on VHS). I want to have all the features of TiVo throughout my home. I want to be able to store a DVD I own to disk and play it back on any TV and any computer of my choosing. I want to be able to download playlists from the media center to a portable music player / iPod. And other things I want to do with this as well.

Originally, I had planned on replacing my nearly 3 year old “gaming” workstation at the end of this year. However, now I am leaning towards getting a media center machine instead. No, I don't think the media center is capable of playing the stuff I would throw at it well enough (regardless of what the manufacturer claims). Maybe I will just push back the gaming machine purchase to later in the year.

Anyways, this is way cool stuff. And it reminds me... I REALLY need to replace my 10 year old 27” television we keep in the bedroom with a new flat-panel. That purchase is just gonna have to wait though I am afraid.

Hopefully Scoble (who knows people at google) or someone who works with Google (such as Adam Bosworth or Joe Beda) will take note of my beef with the new groups site and get something done about it. Apparently scumbag end-users like me just don't have the clout to be heard. Now sure, I could have come across as being less abrasive, but it was 6am after an all-nighter, and cordiality was taking a back seat:

Hi Keith,

Thank you for taking the time to write. Your input is extremely important to us.

As you know, the Google Groups site has been updated with new functionality, notably the ability to create your own group about any topic and invite people to join your discussion group. You can still search and interact with the Usenet newsgroups with which you're familiar, and you can continue to log in with the same email address and password.

While we no longer offer the previous version of Groups, we want to make our new version as user-friendly as possible. If you have specific suggestions or requests, we encourage you to submit a 'Suggestion/Feature Request' at http://groups-beta.google.com/support/bin/request.py. This allows us to easily track which features are most important to Google Groups users. After you click the 'Continue' button, you'll see a screen where you can tell us exactly what improvements you'd like us to make to Google Groups Beta.

We appreciate your assistance.

Regards,

The Google Team

 

Original Message Follows:

------------------------

From: keith@mindfusioncorp.com

Subject: Please revert back to the old groups site!

Date: Mon, 06 Dec 2004 17:58:23 -0000

This new beta site is horrid! What used to take me minutes to locate now takes hours!

I was up at 6am this morning fixing a bug for a down customer, and I NEEDED the google usenet search to work for me... and it failed me! The discussion thread outline is nowhere to be found. The general format/layout of the posts is extremely confusing... was the designer colorblind?? The higlighting of every other word is painful. The seperation between posts is very hard to make out. It also irks me that I cant see a "# of posts in this thread" link on the main search results...

now I have to click a hundred posts that never got a reply before I find one with useful information!

PLEASE PLEASE PLEASE PUT IT BACK!!!

At least make this version of your groups search site OPTIONAL!! I can't stand it!

This site WAS the most valuable research tool I had. Now it just plain SUCKS!!!

 

 

Language: en

Name: Keith Rome

topic: bug

We have a C# meeting tonight in Atlanta. However, the two primary facilitators for the group (Michael and Rusty) don't appear to be available lately (Michael is on assignment in SF, Rusty was MIA last month).

I don't see a topic posted on the website, so I am wondering if there even is one planned?

Microsoft is moving within days, are we sure the meeting will be able to be held as scheduled at the concourse building?

Any of you other folks know anything?

Is this thing on? Hello?...

Why on earth would the folks over at google start redirecting users to the beta groups site when you issue a search from the normal one? Perhaps I am overly annoyed given that it is 6am on a monday, and I have not even gotten to bed yet thanks to troubleshooting a bug. And right now, more than ever, I *NEED* the search for groups.google.com to work correctly. But here they have gone and redirected me to a “beta” site which is incredibly confusing and useless.

What on earth are you thinking??!?

[edit:]

45 minutes later and I solved my problem in spite of google. What should have taken me only minutes to find on the usenet search ended up taking hours.

Change it BACK Google. This “new” version is awful!! Why even change it in the first place? It worked great before this.

The real one this time, not a beta, can be downloaded from Microsoft here.

I just read a news report that says Microsoft is announcing today “New Communication Service That Enables Blogging, Picture Sharing and More”. MSN Spaces is what they are calling this new service.

“The MSN Spaces beta version is a free service available in 14 languages and 26 markets worldwide. MSN Spaces was designed to make it easy for consumers to create and maintain a personal Web site, bringing the power and benefits of blogging to millions of Internet users, regardless of their level of technical expertise. More than a blogging tool, MSN Spaces is a dynamic online scrapbook where consumers can share photo albums, personal music lists and more. And more than an ordinary personal Web site, through seamless integration with MSN Messenger and MSN Hotmail, MSN Spaces will automatically notify online contacts when a person's Space has been updated so his or her online community knows when it is time to pay a visit. People can sign up for MSN Spaces through MSN Messenger or by going to http://spaces.msn.com. Key features of the service include the following:

• Control your Space. Consumers can choose the people who visit their
  Space through three levels of permissions: public, MSN Messenger
  contacts only or private.
• Use pictures and music to say more. MSN Spaces enable consumers to
  easily display their pictures via a photo album slide show. Consumers
  can easily share playlists through their Space with Microsoft(R)
  Windows Media(R) technologies. With just two clicks, people can sample
  or purchase a song on someone's playlist through MSN Music**.
• Create an extension of yourself. Contact Cards - a new addition to MSN
  Messenger and Hotmail - are windows into a consumer's Space, mirroring
  its look and the most recent information posted. MSN Spaces also
  supports RSS, so consumers can publish their Space to others by way of
  RSS viewers and aggregators -- including My MSN, coming soon.
• Post remote updates. Consumers can post updates to their Space remotely
  via e-mail or a mobile phone.
• Make it your own. Fifteen custom backgrounds and five layout templates
  give consumers a way to quickly customize and personalize their Space.“

COOL!