Market Data Trader

Irene Aldridge will be giving a couple day workshop on HFT models, tools, etc.  It’s targeted for Quant Desk Traders, Research Analysts, and PhD students in Finance, Economics, and of course… Math and Physics. 

I am most curious about the content on the “reading ticker tape with the machines / what information is accessible from HFT ticker-tape reading.”  Personally, being trained by someone who traded only from Time and Sales data alone for years, I have a lot of personal intuition about the space.  It would be interesting to hear where my intuitions line up with common industry understandings.

I can save you time if you are curious about cutting edge technologies… build vs. buy.  Of course, buy if you can… and have nothing… and start with Streambase if you can afford it.  For those of you who are interested, more information can be found out here.

If you haven’t investigated RStudio, the positive reviews continue to come pouring in.  I hear kudos for auto-completion, but, still leaves much to be desired on version control, source tidying, plotting, and more.

Check out the 3rd annual Applied Finance with R conference here.  John Bollinger will be speaking at this event.  I had the opportunity to hear him last live in Seattle a couple years back.  That doesn’t perk my interest nearly as much as the tutorials:

• Jeff Ryan’s talk on Algorithmic Trading with R,
• Guy Yollin and Scott Payseur on High Frequency Data Analysis with R
• Eric Zovit on Financial Risk Models with R: Factor Models for Asset Returns and Interest Rate Models.

I would be there if I wasn’t scheduled for vacation

Thanks to Matt at lab49, I’ve found my way to Tabb.  I do recommend the article he referred to on Stat Arb.

There continues to be a general dearth of books on Azure.  At least someone has found a couple here.

Just when you thought you had a good grasp of where technology was taking us… guess again.  I found this to be quite breathtaking.  If you haven’t the time, jump to about three quarters the way into the video and take a look at the future of screen technology.  Truly original, truly brilliant.

Although not directly related to financial markets and software, if you have an appetite for innovation and entertainment, this post is for you.  I’ve never before been so excited about a video game.  I’ve officially pre-ordered my copy of Rockstargames’ “L.A. Noire.”  What’s all the fuss about?

Take a look at the video on the technology behind the game.

Let me paraphrase:  The creators of Grand Theft Auto have made a leap forward in technology that will enable players to exercise their skills in the detection of dishonesty.  The story is set in crime solving in L.A. with an emphasis on interrogation.  What’s interesting here is the increasing capability of modeling technology and it’s application to real world world problems.  For the production of the game, they go beyond traditional motion capture, introducing a new visual modeling ability referred to as “motion scanning.”  In the “depth analysis room,” they shoot two dimensional video from many angles.  This media can then be applied to 3D models.  It is a new form of facial motion capture that provides the players with significantly increased realism.  So, much so, that a core theme of the game is to understand whether people are lying.  Watch out Dr. Cal Lightman!

Rockstargames is not alone in applying the technique of mapping 2 dimensional videos to 3D models.  Take a look at this video from Microsoft Research:

For those of you who will be playing, it may be time to brush up on your applied psychology: interpreting microexpressions, through the Facial Action Coding System, and body language.

Releasing products to the market too late with too little “wow” in them.  Incapable of innovation.  Incapable of design.  Producing products of questionable quality.  These are just some of the common criticisms I hear about the software giant.  But, curiously, I find myself, never more excited in Microsoft technology and it’s future.  Why?

Whether you’re a technologist or an investor, here are a few observations to consider.

Observation #1. Which mega technology corporation has invested heavily in redesigning the developer experience?  Google, Apple, IBM, etc.?  It is no secret that technology is changing rapidly.  Not only is the technology itself evolving, but also how we build it.  The difference between a development environment today and one ten years ago is significant.  Similarly, some languages enable encoding some experiences better than others.  Microsoft has invested significantly in retooling and pioneering new languages.  Who else has done so?  Not just new and exciting languages like F# and significant revving of C#, Microsoft has also made significant investments in resurfacing the development experience around user interfaces and communications with updates such as WPF and WCF.  These investments enable developers to strike quickly.  The deeper impact of these innovations hasn’t begun to hit the market yet.  And, competition has much inertia to become like Microsoft, which remains a business committed to it’s developers.

Observation #2. Which mega technology corporation has such a broad spectrum of platforms to pull together for truly integrated experiences?  Forget “three screen experiences,” consider ubiquitous compute.  You are on the network 100% of the time.  Every theater you visit, every camping trip, every store, every bus, plane, and train.  With mature mobile, search, web, cloud, gaming, and desktop compute experiences Microsoft developers have their hands full making for deeply integrated experiences.  Much of the envelope of innovating experience requires deep integration amongst these platforms.  Companies with an ability to integrate easily will accelerate down the pathway of innovation.

Observation #3. The cool factor isn’t as far away as you may think.  Has anyone noticed an updated image in Microsoft’s brand and software experience?  Windows 7 is a solid hitting OS.  Gone are the days of slow Vista.  Not to mention Microsoft is now even teasing the world with a renewed focus on design and experience.  Take a look at http://www.microsoft.com/design/.  Anyone notice a shift in where Microsoft is connecting with the market?  No longer limited to business, Microsoft is starting to connect with the individual and family in new ways.  Consider XBox Kinect and Windows Phone 7.  Although Microsoft’s legacy reputation of never innovating is still in the air, the business seems primed to shed that old skin for an entirely new image.

Observation #4. The smart factor isn’t as far away either.  I’m a class “A” nerd with more math books than novels and over fifteen years of professional programming experience.  And, having worked with academics and researchers, I don’t have a lot of memories of being “wowed” by the “smart factor” in Microsoft technology.  It has a history of being a mile-wide / inch deep type platforms.  But, with the maturation of BI tools, awesome data platforms, scale out compute solutions, a comprehensive set of libraries and systems, and a renewed commitment to quantitative research, Microsoft is on my map.  Check out, the Modeling the World initiative by Microsoft Technical Computing team.  If you are more into statistics, Matlab, and related tools, you might want to give Sho a try.

If the 80’s were the decade of apple and mac, the 90’s the decade of Windows and Microsoft, the 00’s the decade of OSX and Apple’s return, I suspect the 10’s to be remembered as the decade of Microsoft’s return.  The stage seems set for the pendulum’s return.

I’ve known about Sho for over a year now.  I’m excited that it is available to the world now.  This announcement was made on Soma’s blog shortly after the new year.  For those of you who have a history with Matlab and R, you might want to take a look at this.  For those of you who have been deep in the .NET Framework and are looking for a bit more of a scientific computing experience, this may be a temporary solution for you.

My continued research on “low-level” WPF custom controls lead me to uncover an important subtlety regarding Z-Index support in WPF.  It’s worth keeping this in mind as you set out to design your own “low-level” custom controls.

Yes, there is support for Z-Index attached properties in the major layout panels now, such as Canvas and Grid.  (In the earlier (pre-3.5) days of WPF, you were limited to Z-Index support for children of Canvas layout panels only.  But, those days are ancient history now.)  But, keep in mind, Z-Index support scope is implemented on a per-panel basis.  This means that if you have nested panels or peer panels, all children objects of those panels MUST be either above or below all children objects of another panel.

That’s it!

Just a few of my highlights from the Symposium:

• RX! Linq for real-time events!  Awesome!
• Background worker threads as you know them are dead?!  Look at the Parallel / Async Patterns put out by the patterns and practices team
• On the MVVM front, more confirmation that Commands are losing support.  The rumor has it more and more are starting to just bind to a method.  Makes sense doesn’t it?
• OData everywhere!  In the old 80/20 game, OData addresses 80% of the clients’ needs.  Just stand your data up.  Easy to consume.  Easy to provide.
• Keep an eye on REST.  With Glenn Block joining that effort, you can expect a bang up job to be done.

Hopefully, I’ll get some more time to put out some more morsels from this week long affair.

Things I was supposed to tell you:

• Test driven development is not a questionable practice.  The jury is long in on this.  Everyone benefits.
• More positives on the pair programming and agile front.

I mean to post this back in August.  But, I have some detailed notes from the second in a series of four Seattle Silverlight User Group meetings dedicated to WP7 development.  Looking back, these meetings were gems.  It was really strong Microsoft representation at these meetings including: Pete Blois, Jaime Rodriquez, and Karl Shifflett.  I believe this meeting ran just over two hours.  I think there are still some useful comments in this notes for individuals scrambling to get a perspective on WP7 development opportunities.

Note: for those of you who follow Karl Shifflett, he switched from the Cider team over to patterns and practices and now has more than one blog that he is actively contributing to: the original and the patterns and practices blog.

Also, patterns and practices has put out guidance on WP7 development which should NOT be ignored.

With that… here are the raw notes from August 2nd:

 

Peter Blois and Jaime Rodigeuz will be speaking

• Lead program manager on expression blend

Blend features that Peter covered:

• Sample data

• Window phone styles in blend — so you get the look and feel of the phone off the bat in blend
• There is a number of built in styles for text in phone
  • You are expected to follow the look and feel in phone — the metro style
  • You will see built in "OS" fonts (built into the phone)
  • There are "themes" in the phone that blend supports, such as "the blue theme" and "green theme"
    • Themes cover specific uses of color
  • There is a light vs. dark theme that will invert colors
  • Instead of taking explicit colors, take from the system resources

• This is important especially in text where you go from light to dark and dark to light
  • (assuming you will support themes

• ANIMATION
  • Seamless transitions, seamless interactions should be very easy
  • VSM states — visual state manager
    • Allows you to define different states of your UI
    • He demoed how to do the page flip
      1. Set the center popint of the rotation
      1. Then set two states in the VM: in and out
      1. Then do the rotate on the y axis
      2. And set the opacity to invisible when it is fully "out"
      1. Set the transition time to about half second
      1. In code just call: goto "out" state
      2. You can also give a little easing to make it a little cooler
    1. If you aer doing full on page transitions –
       1. Look on the web
          1. Transitioning content control samples and just follow Win7 look and feel guidelines

• Another thing the transitions are cool for is orientation support
• Blend has landscape to portrait feature
• If you double click the button and auto set it, it will pick up the recommended style size for the buttons
• Don’t forget to turn on orientation support if you are going to support orientations

• To use transition with orientations:
  • Create two states: portrait and landscape

• BEHAVIORS
  • Drag drop interactivity
  • Team is really excited about this and likes to use this
  • Example: GoToState
    • You can just wire up the controls
  • Example: Orientation
    • Just listens for orientation event
      • Blog samples: look for food samples
        • 20 minute PDC talk — all code is available
        • Kenny Young / food layout / animations
      • This stuff will blow your mind for trying to get animations in silverlight

• Question: can we change app title sizes or not use them?
  • Pivot and panarama controls you can’t change… they don’t use titles… but, you can’t change the style… that is controlled
  • Those controls are coming.
  • Look at applications that are out there on codeplex and see what they are doing

• APPLICATIONBAR
  • Something they built into blend to make it really easy to add
  • Click in the control / properties tray o Application Bar and then "new" and customize it and can add buttons…
    • There is the list of "suggested" icon button, such as the + and menu items
  • Question: is there any databinding in the applicationbar?
    • No. It’s a platform limitation. We can add it later when we can overcome that limitation.
    • Applicationbar is used on the phone in XNA apps and they don’t have the concept of dependencyproperties, etc. It’s the OS’s application bar.
    • There are blogs that use RX or other APIs that make them look databound, but, they are not. There are only four buttons, so, it’s not a lot of code to support them.

• MISC IDEA: become a control vendor and provide BLEND support for your control libraries

• TIP: RE ANIMATIONS: if you are doing animations, animate doing render transforms. Do storyboard animations as much as possible. (Render transforms or perspective transforms)… stick to storyboards — states are animating storyboards

• Question: how is the behavior support for drag and drop?
  • Laurent bugnion has the code for multi-touch rotate pinch, translate, rotate
  • You are dealing with gestures or raw touch input
  • Rotate has to deal with raw touch input

JAIME —

"the software session"

If you are going to write an application, what are the new concepts: "navigation", etc.

Next month we will look at the hardware such as accellerometers

Beta2 is pretty code complete… if you are going to write an app, beta2 is a pretty good place to start. (beta2 is the july release)

Three things missing

• Missing some core controls
• Tombstone install is hopefully going to get a little better
• The rest of it is pretty good, except for bug fixes
• They will not cut a lot of things here on…
• Runtime is stable — a lot of performance improvements

• Uninstall is getting better

Maybe next month some of you will have phones…

Last time recap:

• Silverlight 3+ optimized
• New controls on the phone
• Navigation and navigation models

• The controls on the phone are the same controls that are the CORE SILVERLIGHT controsl… (the 5meg download… not the SDK… datagrid, etc., not the TOOLKIT, which is ms people writing controls that are still churning)….
  • Most of the SDK controls don’t make sense on the phone
  • In the future, they will release a control toolkit on the phone
    • Wrap panel, view box
    • They have the code in-house… if you need it, email them, they will hook you up.

• InputScope gives a hint ot the system which Soft Input Panel layout to use
  • <TextBox>
    • <TextBox.InputScope>
      • <InputScope>
        • <InputScopeName NameValue="EmailNameOrAddress"/>
      • </InputScope>
    • </TextBox.InputScope>
  • </TextBox>
  • This is an optimization done on textbox
  • Use it if a user is going to input a specific type of entry, such as numbers or email address and you will get a slightly different keyboard
  • Examples: Chat, EmailNameOrAddress, CurrencyAmount, PostalCode, Digits

• Also password box, was optimized with the 1 second delay so you can see the character before it’s masked

• Keyboard is shown whenever the control has focus…
  • Set focus onto something else, if you want to programmatically hide the keyboard

• WebBrowser control is new…
  • It just brings HTML or web content into your application
  • If it has javascript, you can communicate back and forth between your C# and javascript
    • ScriptNotifyEvent — is event used to contact C# from JavaScript
    • BrowserControl.InvokeScript method used to call JavaScript from C#
  • Displays network and local content
  • Gesures: pan, double tap and pinch
  • Supports transforms & projections
    • No input on transform
  • Optimized for privacy
    • Script is disabled by default
    • Cookies and cache are isolated

• The application model on the phone works like this
  • You have one application, with one isolated storage, and own http connection
  • It’s all isolated
  • The cookie set by your app logging into facebook app is different than the cookie and cache and so forth set by the main browser on the phone
  • Everything for your application is separate
  • It was a design choice they made

• Paradigms for navigating:
  • For most applications we expect to navigate through PANORAMA or PIVOT
  • "Panorama" — takes one page….
    • Turns your application into a really fun to navigate, interactive page
    • We go "broad"
    • Put three screens or more and the user navigates across the pages with a flick
    • Good for "exploring" content
    • This will NOT have an application bar
    • These are ALWAYS portrait — we WILL NOT support landscape
    • Has TITLE at the top
    • Has SECTION titles
    • Panorama title moves at different speed than the section titles
    • They do all that for you in the control
    • It always give you a little "peak" of what is coming next, a teaser
    • These things go in a circle, it’s a real that turns around… they do all that for you… you just give them the IM’s
    • It’s not in beta2… they have it in house… if you need it email Jaime or Karl or whoever
    • You can scroll up or down… still will support deep zoom or pinch gestures
    • It should be 6 or 7 screens tops — then the user gets tired scrolling back and forth
    • 4 panels seems really straightforward
    • Question: programmatic access to go to a specific panel? Take me home or goto page 4…
  • "Pivots" — horizontal list with a list box down
    • It’s just like a tab control
    • You have tabs on the top and you press on it and it will bring in new data
    • Zune HD does pivots for just about everything
    • This is not in beta2… it will be in the upcoming release once they RTM
  • There are codeplex examples out there both Panorama and Pivot
    • You can play with those…
    • But, there are some significant differences… so wait for them

• BING MAPS CONTROL
  • Not in beta bits
  • Can use it inside your app
  • In-App bing maps control
  • Use it to show maps and data and pins
  • It has gesture support for zoom, pinch, flick
  • Programmable
  • Show pins, hide pins, replace pins
  • You need an APPID to use it, but, it won’t be a paid license, it will be free
  • For most developers it will be free… if you have a real-estate app…
    • You used to have to license it
    • You don’t have to do that… just get your app ID and code against it

• THEMES
  • Their controls support themes
  • Select from a choice of 10 accent colors
  • Select between light or dark background
  • Many applications won’t support the themes and that’s ok.
  • If your application is simple… and you support the OS, you’ll get theme support out of the box

• Question: standard support for ResourceDictionary is there
  • Just like WPF, walks up the tree looking for themes, if it doesn’t find it, it goes out the top and grabs the system themes

• APPLICATION MODEL + NAVIGATION:
  • "how is my application going to work / what’s the difference between the app I have and what I’m supposed to do outside of the desktop"
    • Phone has three buttons: back button, home button, and search
      • There is little to nothing you can do with search…
        • It’s for bing
        • Might be opened up later
      • Back button is interesting, you can use it to navigate the phone and in your app
        • Example: of a wizard app with a "goto next step button"
          • Asks survey… in your code you have a button that will programatically take you back
          • BUT the phone back button you can use to navigate back too.
          • Integrates seemlessly with the phone and the OS
        • SILVERLIGHT FRAME AND PAGE REFRESHER
          • You have a "frame" object on the phone and a "page" object… this takes care of all the navigation
          • The operating system takes care of all that for you
          • Frame is really just a dummy object
          • Page is really a more interesting object — has a bit more weight… there are some events at the page level that handle saving state… when they leave page and come back
          • Standard silverlight stuff that they are using on the phone
  • APPLICATION CHROME
    • SYSTEM TRAY and APPLICATION BAR
      • SYSTEM TRAY
        • Your application has the client area and you may have a System Tray up above that…
          • It’s like the windows task bar
          • You can NOT interact with it
          • You can NOT put buttons in there
          • ONLY windows and operators for things like battery
          • The one thing you can do there is to HIDE or SHOW it
            • It’s just an attached property
              • Microsoft.Phone.Shell.SystemTray.IsVisible = true
              • Design your app so it fits when the tray is visible
      • APPLICATION BAR
        • It’s sort of the menu on the phone
        • It’s always on the button
        • They always handle it
        • You can’t control the animation on it…
        • You CAN control the background color
        • You can have UP TO (but no MORE than) 4 buttons
        • Icons should be one color — they will overwrite and apply a mask on your button colors
        • From the application bar you can have MenuItems (click on the …)
          • Logical limit is 6 items
          • Although there isn’t a hard limit
        • Automatic support for rotation on buttons and menu items
        • These 4 buttons should be pretty "global" — things that should be global in your APP — NOT context sensitive things… like "home" or "refresh"
          • If you want to do something context sensitive… you might look to the menus

• ORIENTATION SUPPORT
  • You can NOT programmatically change the orientation
  • PhoneApplicationPage.SupportedOrientations property states Portrait, Landscape, or Both

• WP7 APP MANAGEMENT
  • Windows Phone execution model is designed to provdie end users with a fast, responsive experience
    • Only one application can run at a time
    • System terminates application when user navigates away
  • It’s a multiasking phoen (three layers underneath it’s WindowsCE)
    • But, higher up… it’s ONLY one app at a time
  • App is TOMBSTOED when user navigates away
    • System saves state information then terminates your app
    • When user navigates back to the application, system restarts the app and passes state information back
  • It’s how you restore your state that makes the illusion that your app didn’t get shut down
  • Developer must write code to respond to lifecycle events
    • Save and restore state
    • Maintain illusion that app wasn’t stopped
  • RUNNING -> EXIT -> NOT RUNNNING -> START
  • Interruptions: lock screen, low battery, app switch, reminder, text, phone calls… (these are things that will interrupt your application
  • Approach #1: Obscuring
    • Scenario: Playing game and phone rings
      • Dims and a little bit of a pop up on the phone…
      • "hey, you have something to look at"
      • Your game is still the main thing that is running…
      • You still have the main phone
      • If you are playing a game… your game should automatically PAUSE — so you don’t get "shot" in your game
      • All the background threading is still running
      • All that happened is that your UI got obscured a little bit
      • If the user took the call then…
  • Approach #2: Tombstoning
    • Scenario: you were gaming, incoming call, you click on take call
      • Your app is toast
      • You get an event called "Deactivate"
      • This is what you get before you get into "Suspended"
        • Suspended == dead… your app is dead
        • It’s called suspended because in the future eventually they will open that up a little bit
  • Approach #: Activate
    • If they click the back button, they are activated again, your app is live
    • To the user it looks like they never left
    • When they were deactivated, you saved all the state, the viewmodel all of it… they give you a property to store all this
    • Then they kill you
    • When you come back, you get this property back and you restore all the state and everything that is going on…

• Events:
  • Application_Launching
  • Application_Activated
  • Application_Deactivated
  • Application_Closing

• There is no "exit" code in the phone.

• You don’t call return or Exit

• If the user clicks the back button, you are dead, app closed.

• The other time your app will die is if a user clicks on an application

• Look at sample app:
  • Joie kyriorte on jaimie’s team cleaned it up
  • And shows "App Lifecycle"
  • "Learning Windows Phone – App Lifecycle"
  • It’s out on the web
  • It shows all the event log in the body of the application

• In the emulator you can not simulate a phone call…
  • But you can use "PushNotifications" or something to create an interruption to simulate the Obscured state

• One example that they are still working through…
  • Rebuffering of a youtube video with tombstone

• Question: how much time do you have… still being debated… rule of thumb is about a second or 2… you won’t be able to extend it. (it will be between 1 and 5)…

• Question: saving state is not done in background
• When Deactivated is called:
  • #1: kill any running threads
  • #2: save your state
  • – don’t do any UI
  • – don’t save your state on the internet
  • – the will kill it
  • – save locally as much as you can
    • When you come back you can get at it and save it out to the internet

• If you launch your app from the home page… it’s a new instance
• If you launch your app from the back button… you get the instance on the "back stack" — it won’t have the prior state

• You can have two instances of the same app…
  • You can use isolated storage OR ???
    • You can save to either place

• When you are on your main page and you click the back button…
  • That is the one time that you have a closing event.

• There is a great blog post on Tombstoning

• THERE IS NO CONCEPT OF FORWARD…
  • There is a back stack that keeps growing…
  • There is no forward
  • They don’t allow you to control the journaling yourself…
    • On the internet you can make them think they went from 3 to 1

• Scenario 1: Launch and Clsoe
• Scenario 2: deactivated > user completes choser or navs back
  • Then app is restored and Activated Event

• LAUNCHER or CHOOSER
  • These is how you interact with the phone OS
  • LAUNCHER
    • Launches on of the built-in applications through which a user complets a task
    • No data is returned to calling application
    • Example: PhoneCallTask
  • CHOOSER
    • Launches one of the built in apps through which a user completes a task and which returns some data to calling application
    • When caller completes, calling application is activated and supplied with the Chooser result
    • Example:
      • Common file dialog
      • Want to choose a picture… go choose the picture from facebook
  • These are very simple
  • Launchers
    • Emailcomposetask
    • Marketplacedetailtask
    • Phonecalltask
    • Searchtask
    • Mediaplayerlauncher
    • Webbrwosertask
  • Choosers
    • Cameracapturetask
    • Emailaddresschoosertask
    • Phonenumberchoosertask
    • Photochoosertask
    • Saveemailaddresstask
    • Savephonenumbertask
  • If it ain’t on the list, you can’t do it.
  • Example: how do I call office? If it ain’t on the list for V1, you can’t do it.
  • These make tombstoning a little more complicated
    • When you launch a Launcher or Chooser your app is terminated.
    • Chooser
      • Assign a dleegate for the Chooser’s Completed event
      • Implement the event handler for the Completed

• HARDWARE AND THE CLOUD IS PART #3… next months session…

Did I mention I’m in the process of wrapping up a white paper on the quantitative finance industry.  White papers are fun to start and tedious to finish.  The conversations around the paper have been interesting.  Yesterday I discussed with a Quant, how often individuals use factor models to value equities.  From our perspective, the answer was not very often.  This is interesting, considering how common factor models are.  But, all the Quants that I know would prefer to do some custom statistical pattern recognition.  That’s not to say we are hacks.  We’ll still use a Monte Carlo or bootstrap to add a layer of rigor around what we are doing.  One exception that was brought up was Arrowstreet Capital.  I’m pretty sure they don’t qualify as “Quant.”  But, rumor has it that they ARE on the record for using factor models in this fashion.

…noteworthy from yesterday, the Tech Tock blog posting on RX that surfaces through lab49.  That was my first introduction to RX – the reactive framework by Microsoft, which is focused on programming patterns that can make your program “react” to it’s environment.  Very cool / interesting… beautiful to see patterns for marrying Complex Event Processing (CEP) type streams with LINQ.  I think you’ll quite like it.

…spent a little time looking at CPLEX.  Not sure how commonly used that is.  But, because it’s on my radar, I’d argue it is a contender in the Quant optimization space.