Flex Monkey Patches

I had an opportunity too look into eking out every last bit of performance on an LCDS RTMP channel and came across this property:

socket-tcp-no-delay-enabled

The docs that describe this stuff is here:
http://livedocs.adobe.com/livecycle/8.2/programLC/programmer/lcds/lcconfig_4.html

The docs seem to say that the “socket-tcp-no-delay-enabled” setting is a property of an NIO socket server.
The section that describes configuring NIO RTMP seemed wrong to me or at least has me confused.

in the LCDS install dir: resources\config\services-config.xml (this is a good additional resource for configuration parameters)

this comments say:

Which don’t really help clarify things for me.

After speaking with Seth and Jeff from the LCDS engineering team, they were able to set me straight (and then some ).

Socket settings are up to the OS. Adjusting that is OS dependent and I don’t have any steps to offer offhand.
The default for this socket setting is false (meaning the Nagle algorithm is enabled) on various platforms that engineering investigated (Mac OS X Version 10.4.10, Windows XP Version 2002 Service Pack 2, Linux 2.6.9-24.ELsmp). The Java Socket API let’s you tweak these (but in some cases, such as send/receive buffer sizes, invoking a setter is just a request/hint that may be ignored by the platform). It’s best to configure these in LCDS config, so your desired settings are only applied to sockets that LCDS is managing.

When a Socket’s no-delay is false, this means that the Nagle algorithm is enabled. By overriding the default, setting this to ‘true’ in config, you’ll disable the Nagle algorithm.

The Nagle algorithm was designed to help with congestion avoidance for TCP-based applications like Telnet, which may send a single character per TCP packet, leading to lots of unnecessary packets and overhead. With the algorithm enabled, as long as there’s an outstanding un-acked TCP packet the sender will buffer any further bytes to send until it has a full packet’s worth or until it receives the outstanding ack it’s waiting for. So you get something like send – buffer –buffer –buffer – send – buffer … So with Nodelay on, the latency of small messages is reduced but overall throughput probably goes down a bit if you send lots of small messages (i.e. less than a packet size). With nodelay off, smaller data writes (i.e. less than the MTU) are held so they can be combined with other small writes.

Both RTMP and HTTP are just bytes over a TCP socket, so whenever RTMP data or an HTTP response is written back to the client, this algorithm will come into play as the bytes are put on the network.

This is where the docs where incorrect (and they are being fixed as we speak.)
In order to configure a RTMP channel to set the socket-tcp-no-delay-enabled property you would do something like this:

HTH

-Kyle

OK…so I guess I kind of passed over these yesterday in my announcements. To me, the fact that there would be new releases timed with MAX was known and to anyone in the community there was probably a strong suspicion that there would at least be a new point release at MAX 2008. But in an effort to correct my oversight, here are links to the announcements by the product managers:

Matt Chotin (PM for Flex SDK) announces Flex Builder 3.0.2 and Flex SDK 3.2 (with support for AIR 1.5 and Flash Player 10).

Anil Channappa (PM for LCDS and BlazeDS) announces LCDS 2.6.1 and BlazeDS 3.2.

Kind of related to an earlier post: Deploying Multiple LCDS wars to the same servlet container

I found out that you can specify config file tokens as JVM args. This makes deploying the same war file to various deployments that have slight variations for certain parameters easier, since you can deploy the same war, but “tokenize” (and/or use some of the default tokens).

http://livedocs.adobe.com/livecycle/8.2/programLC/programmer/lcds/help.html?content=lcarch_4.html

“You can also use custom run-time tokens in service configuration files; for example, {messaging-channel} and {my.token}. You specify values for these tokens in Java Virtual Machine (JVM) options. The server reads these JVM options to determine what values are defined for them, and replaces the tokens with the specified values. If you have a custom token for which a value cannot be found, an error is thrown. Because {server.name}, {server.port}, and {context.root} are special tokens, no errors occur when these tokens are not specified in JVM options.”

I know Damon Cooper, Matt Chotin and Tom Jordahl have all posted links to this:

Adobe LiveCycle Data Services 2.6 Capacity Planning Guide

but what they failed to do is tie it in with the links below in the LCDS 2.6 docs:

Now it is tied up in a nice, neat package…

-Kyle

If you look at a standalone install of LCDS 2.6 you will see 2 instances of LCDS deployed in the 1 Tomcat Servlet container.

\tomcat\webapps\lcds
\tomcat\webapps\lcds-samples

These 2 instances “should” be configured so they can run simultaneously without any conflicts.
(I say should since these are really meant as samples/demos and conflicting ports may not have been taken into account since for demo or dev purposes you could easily remove one of the deployed wars should there be any detrimental conflicts.)

The key to having 2 LCDS war deployments in the same servlet container is within the configuration of your endpoints/channels within the services-config.xml file.

Looking in the sample config files located here: \resources\config there are a few useful comments in the services-config.xml file that are useful to note in answering this question:

This essentially means that endpoint urls that look like this:

http://{server.name}:{server.port}/{context.root}

will have their tokens “filled in” at request time from the servlet container.
(ie when you request the swf via http://myserver:8080/samples, the server.name, server.port and context.root tokens will be filled in for the endpoint from the URL used to call the swf.)

So you either need to stick with using the tokens in your config or hardcode unique endpoint urls per LCDS war deployment.

Endpoint urls that look like this:

rtmp://{server.name}:2038

will have the server.name token “filled in” at request time. 2 LCDS wars deployed in the same servlet container will have the same server.name token and so must be configured to listen on different ports for each deployed war.

Another useful comment in the services-confi.xml file is:

So you will see endpoint urls that look like this:

http://{server.name}:2080/nioamf

In this case, like the rtmp endpoint, the server.name token would be “filled in” at request time. 2 LCDS wars deployed in the same servlet container will have the same server.name token and so must be configured to listen on different port/ combinations for each deployed war.

The docs on LCDS Channels and Endpoints can provide you with more info:
http://livedocs.adobe.com/livecycle/8.2/programLC/programmer/lcds/index.html

Also for a good summary of channels/endpoints see this blogpost by one of the LCDS engineers:
http://www.dcooper.org/Blog/client/index.cfm?mode=entry&entry=8E1439AD-4E22-1671-58710DD528E9C2E7

This seems to be a little known fact.

After installing LCDS, you can go to the resources\config directory and find heavily commented config files. The comments are a great additional resource in deciphering all the various ins and outs of configuring out your LCDS application.

Take a look…

I beleive in the “80-20 rule”.

http://en.wikipedia.org/wiki/Pareto_principle
http://management.about.com/cs/generalmanagement/a/Pareto081202.htm

I think it certainly applies to solving the types of problems I face day in and day out as a part of my job.
Within this realm, I see its application as thus:

For a known set of questions (related to a given topic), 80% of the solutions have already been found (or closely related solutions have been found), leaving 20% of the solutions yet to be discovered.

If there are others in the community who are publishing questions and solutions, then I should be able to find the solution(s) (or other solution(s) that are close enough to make the jump to the one I am looking for relatively trivial) to 80% of the problems I encounter.

Anecdotally, I find this assumption to be not too far off.

So, over the course of assisting customers with Flex/AIR/Flash/BlazeDS/LCDS related problems, I often perform quite exhaustive Google searches on particular topics to see if I can find information to solve or help solve a problem. I come across all sorts of info during these searches. Some of it useful, some of it not.

On one recent occasion, I found I was coming across a lot of good info on the broader topic I was searching on, but it took me a while to find out what I was specifically looking for. I did eventually find something that solved or helped solve the problem, but rather than discarding all the other info that I had gathered, I thought it would be better to collect it together and post it somewhere that would make searches that fall under that broader topic more efficient.

This idea essentially resulted in me devising a new set of pages for my blog. Each page will essentially be a collection of human-indexed topics related to Flex/AIR/Flash/BlazeDS/LCDS. During my searches for specific solutions, I will try to remember to keep the other related results (although they may not solve the specific problem I am tackling) and post them into a more general “topic related” page.

The “jumping off point” for these pages will be an index located here:

The Monkey Index

I already have a few pages to add and will do so later this week and (hopefully) continue to do so as I, one monkey, attempt to index Google for Flex/AIR/Flash/BlazeDS/LCDS related solutions.

This really isn’t that difficult to do, but the docs regarding it are a little hard to find. I thought I’d post the link and then some links to other useful blog entries that have info to add on this topic.

Doc link: http://livedocs.adobe.com/blazeds/1/blazeds_devguide/help.html?content=runtimeconfig_5.html#194376

One of Adobe’s platform evangelists has posted a blog entry that may be more clear:
http://raghuonflex.wordpress.com/2008/06/05/endpointatruntime/

There is also a “Flex cookbook” entry that may be of some use:
http://www.adobe.com/cfusion/communityengine/index.cfm?event=showdetails&productId=2&postId=9703

Here are some debugging tips I provided to a customer this week regarding debugging ant tasks that seemingly compiled a Flex app that used dataservices fine, but the app didn’t seem to get any data. The are generally useful, so I thought I would post them.

1. Try setting fork=”false” in your mxmlc and compc ant task, since forking the compile process will hide the stdout and stderr streams from the compile process and hide any errors or warnings that may be happening in the compile process, setting the fork to false will allow you to see any compile output in your console.

2. In your ant target echo out a formatted statement that would represent the equivalent mxmlc (or compc) commandline statement with all variables and relative paths, etc. resolved. Then you can use this echoed statement to run against mxmlc to help detangle yourself from ant to rule that out as a source of problems or rule out any pathing issues.
(Maybe you are not pointing to correct sevices-config.xml file?)

3. Make sure your dataservice tags and dataservice operations have fault handlers. (Maybe faults are being returned from the dataservice or the operation, but they are not being handled.)

4. Compile the Flex app with and run in the debug Flash Player. This should produce logging info in the flashlog.txt that is helpful for watching the network traffic between your Flex app and a server and may help narrow down the issue.
http://livedocs.adobe.com/flex/201/html/wwhelp/wwhimpl/common/html/wwhelp.htm?context=LiveDocs_Book_Parts&file=logging_125_10.html

5. Look in your LCDS install dir under resources/config at the services-config.xml. This file is a sample config file that has comments explaining the various settings. You want to look at the logging tag and enable more verbose serverside logging.

You should probably set the logging tag like this:

(This will log “Debug level” output to your servlet container’s logs.)
The default patterns are probably sufficient to start with, but this sample config file lists all the possible debug patterns. If the debug level does not result in enough info, you may want to try the “All” level.

6. Use a network traffic sniffing tool like Charles: http://xk72.com/charles/ to view traffic between browser and server.

Adobe Flex sdk beta 3 and Adobe AIR beta 3 just release to Adobe Labs yesterday, so I have updated my version checker AIR application to detect the new SDK and also compiled the app against the new AIR classes targeting the new AIR runtime.

All other blog references will be pointing to this new version

>>>> here <<<<<.