Friday, May 29, 2009

Agile software development and agility

I'm trying to figure out if agility is different than agile software development. To be honest, I've heard ASWD tossed around before, and it has been used as a growing buzzword for a few years now. I still don't quite know what it is.
"Agile methods generally promote a disciplined project management process that encourages frequent inspection and adaptation, a leadership philosophy that encourages teamwork, self-organization and accountability, a set of engineering best practices that allow for rapid delivery of high-quality software, and a business approach that aligns development with customer needs and company goals." [Wikipedia]
My initial thoughts are that the concepts of agility are not necessarily the same as the principles behind agile software development. Agility to me seems to be a property, something that can be stated. ASWD seems to be a process, with methods, that can be applied as a strategy.

Where did ASWD come from? There is a manifesto for agile development back from 2001. Alister Cockburn has a book titled Agile Software Development from 2002 that is highly cited (draft version 3 available here) according to Google. It addresses communication and rules for successful projects.

So then, what is agility in a software product? According to the book, "an agile process is both light and sufficient". That definition is applied to the process, not the software itself. Their example is that a 40 person team is not as agile as a 6 person team, which makes sense. I think applying the term agile to the product itself leads into a lot of what SaaS and SOA is attempting to accomplish. But again, those are paradigms, processes, shifts in development strategies.

Reflections on Software Agility and Agile Methods: Challenges, Dilemmas, and the Way Ahead, by Linda Levine, compares ASWD with differing processes such as the Capability Maturity Model CMU and the Spiral loops from Barry Bohem. Even though the paper is talking about Software Agility, it still seems to be talking about the process of generating the software, not an aspect of the software itself. For agile software I keep thinking back to our computer security class here and the definition of "good software" which was high-cohesion, low-coupling, modular, etc. Do those properties overlap with what an agile component would possess? I think its the best start to defining an agile component (if such a definition doesn't already exist).

Also from wikipedia, is this section (currently marked for deletion citing a strong bias and weasel wording):
Proponents also argue that process-oriented methods, especially methods that rely on repeatable results and that incrementally reduce waste and process variation like Six Sigma, have a tendency to limit an organisation's adaptive capacity (their "slack"), making them less able to respond to discontinuous change - i.e., less agile. It is proposed that "agile", "lean" and "evolutionary" are strategies that need to be properly understood and appropriately applied to any specific context. That is, there is a time to be "agile", a time to be "lean" and a time to be "evolutionary." Some agilists agree with this position, promoting the concept of agile methods as one set of tools that should be available to managers for use in appropriate situations, not as one-size-fits-all methods that should be forced onto all organizations. [link]
Its interesting. From that viewpoint, we wouldn't want a system that is necessarily agile, or one that is evolutionary, but some type of hybrid. The reference for that entire section is:
"Managing Agile Projects." Ed. Kevin Aguanno. Oshawa, Ontario: Multi-Media Publications Inc., 2005. ISBN 1-895186-11-0.

---

You have to go to the aerospace industry before you see the term agility applied to a tangible product. The following is taken from "Flexibility in system design and implications for aerospace systems":
Agility is another term related to the ability to respond to change. It was first introduced in manufacturing environments then broadened to encompass the extended enterprise. It is often loosely defined, and used to characterize different things in a business environment. For instance, in Pathways to Agility, Oleson [12] describes “agile strategic planning processes”, “agile automation”, and discusses the need for “agile business relationships” with suppliers and customers. He defines agility as the “ability to respond with ease to unexpected but anticipated events”. Similarly, Fricke et al. [13] define agility as the “property of a system to implement changes rapidly”, and flexibility as the “property of a system to be changed easily and without undesired effects.” “Agility” is thus used as a desired qualitative attribute for an enterprise to thrive in a hyper-competitive environment. It is difficult however to see how the definitions of flexibility and agility provided by Fricke et al. [13] differ or overlap, and to grasp the concrete content of “agility”.
13. E. Fricke, A. Schulz, S. Wenzel, H. Negele, Design for changeability of integrated systems within a hyper-competitive environment, in: INCOSE Colorado 2000 Conference, Denver, March 2000.

Disasterhelp.gov | OPEN Special Interest Group (SIG)

Disasterhelp.gov | OPEN Special Interest Group (SIG)
Via the OPEN email listserv:

As a matter of interest, I will be presenting a one hours web seminar covering the steps needed to build software that can create a HazCollect Message for transmission to the NWS via DM-OPEN to the DM-OPEN Special Interest Group (SIG) using Microsoft Live meeting on May 20, 2009 from Noon Eastern time. This presentation will be useful for developers and and project managers interested in building to the DM-OPEN Non-Weather Emergency Message interface in order to broadcast Alerts through NOAA Radio. It will describe identifiable milestones for developement, as well as the steps needed to get approval for customers to use a newly developed HazCollect alerting capability.
From the Disaster Management Interoperability Services DM-OPEN group, part of the OPEN Special Interest Group (SIG). I know its not directly related to our research, but it has some overlap with regards to the OPEN Web Services stuff, and in general the standardization of interoperability.

Notes: taylor slide on decentralization.pdf

Unknown, just a single slide: "SOAs, Decentralization, and the Web"
Taylor (Institute for Software Research)

Notes: medvidovic ownership.pdf

Source unknown, just one slide
N Medvidovic
  • Decentralized ownership
Thoughts from earlier about ownership: dFed participant vs. dFed provider.

The terminology here is complex and needs to be elegantly delivered otherwise it will be totally lost on the reader.

In my head dFed participants take part in the formulation of the dFed. That is, they select which providers to include, what the workflows are, what tasks and objectives are a priority. They choose the providers.

The dFed providers expose their services for inclusion in a workflow. Providers also have concerns for things like QoS and security and availability, but we refer to those as contracts to delineate the difference from the participant concerns. Contracts are enforceable (they are being adhered to or not), participant concerns may not be so clearcut.

With that mindset dFed participants own the federation, and dFed providers own the services. Of course there are exceptions. For example, a specific type of service may only be exposed to the dFed (its a secret or a time-sensitive service only available for a crisis). It's still owned by a specific provider, but its created for the dFed participants.

Notes: jackson secure cloud.pdf

Secure Cloud Computing: An Architecture Ontology Approach
K Jackson (Dataline)

  • Motivations for using cloud computing: "New market growth, new approaches, competative pressure"
  • Number 1 benefit from their research: Easy/fast to deploy. For most cases I'd think that is from a business perspective, the ROI-like mindset. While it could be true for a dFed I think its not the same kind of east/fast that we're looking for.
  • Number 1 challenge/issue: security. (then performance and availability). More justification for us in targeting those challenges in the dFed.
  • SaaS (Software as a Service), but then.... PaaS, IaaS, DaaS, CaaS, HaaS.... really?
  • In their "Unified Ontology" a cloud application contains the cloud software environment and the cloud software infrastructure? That figure is slightly confusing. I'm really not sure what the difference between the infrastructure and the environment is.
  • New term (for me at least): "Tactical Cloud Computing", seems to be built into the cloud infrastructure maybe?
  • "Federated SOA" with a concept for "Global Governance" and "Dynamic Tasking"
  • They've got a SecureParser that splits data (documents, video, email, map data, etc.) into different fault tolerant shares. How does this occur? Is it beneficial? I thought most grid computing still required information to be passed back and forth. How does each unit of data get used effectively?

Notes: froehlich cloud tutorial from business perspective.pdf

Cloud Computing - A Tutorial Introduction
K Froehlich (The Aerospace Corporation)

  • Taxonomy of cloud computing (I had an idea for a taxonomy of modern web services that this may be related to). Infrastrcutre, Platform, and Applications. Author of taxonomy is Peter Laird, click here.
  • More a process than a technology, a framework built on top of cluster computing, utility computing (what is this?), grid computing, and on-demand computing (don't know this one either).
  • Goal of cloud computing: "the virtualization and abstraction of resources", -- I disagree with the broad definition but the refinements on that statement are more accurate, because I think that statement alone is the same goal as the semantic web and other like ideas
  • The problem we're going to face: "the push towards open standards for cloud computing is just getting started".

Notes: 11e_outbrief SOA paradigm.pdf

Architecture-Centric Evolution
Ground System Architectures Workshop
M Hutchison (The Aerospace Corporation)

  • Paper is on SOA terminology
  • SOA is a paradigm not a tangible entity, "How do you standardize a paradigm?"
  • Suggests using SMaRT for legacy migration into SOA, Could that be this? (Service Migration and Reuse Technique from SEI)
  • Lots of focus on business-side of the problem. "Agility in SOA framework is more important than immediate ROI". From a dFed standpoint that mindset can kind-of be adapted to an idea that initial agility may prove beneficial instead of a hard-coded workflow that is coded to perform one task very well, since needs may vary greatly over time.
  • References Oasis as a reference model, for what though? SOA?

Notes: 11b_outbrief human systems integration.pdf

Human Systems Integration: Tools, Techniques, and Challenges Ahead
Ground System Architectures Workshop
S Dawes (The Aerospace Corp.)

  • HSI = Human Systems Integration
  • Main challenges: increasing (information) demands, want to reduce manpower
  • "Human factors engineers" - developers tasked with joining programs at the end of development
  • Workaround of workarounds -- a major challenge (how do you manage that?)
  • Automation both eliminates errors and introduces new errors
  • They identify a need for "clear, unambiguous, testable requirements"

Notes: Huang-abstract-veiled certs.pdf

Multi-dimensional credentialing using veiled certificates: Protecting privacy in the face of regulatory reporting requirements

C.T. Huang, J. Gerdes Jr.

Full version of paper here.
Not sure where the extended abstract came from.
Powerpoint slides here.

Questions to keep in mind while reading: What is the accepted research? What does everyone in security policy research reference?
  • Addresses the problem of individuals losing control over their personal identifiers (PI), which is basically sensitive data they'd like to keep secret.
  • Seems very data-oriented
  • Introduces multi-dimensional credentialing
  • Introduces a veiled certificate (VI) which provides individuals control (dFed user concerns?) and satisfies regulatory/reporting needs (dFed security policies?)
  • Public and private key pairs, probably something similar to Skipjack from Computer Security class.
  • Has a VC creation protocol
  • Interesting organization strategy, they fully describe their work up front then do a comparison with other forms of digital certificates after that.

Notes: Hafiz-abstract- RJohnson- component replacement.pdf

Security Oriented Program Transformation
by M Hafiz, R Johnson

An extended abstract.

  • 42 Security Oriented Program Transformations, available security solutions that can be applied to a system after it has been created
  • Based on vulnerability analysis (using CERT's vulnerability metrics)
  • They use their own catalog of security patterns [13]
  • Paper is interesting because in it they are not just talking about their catalog, but attempting to validate the catalog
  • They describe their 3 step process for generating the program transformations. I like that they don't just show the transformations but they describe how they generated it. (1) Selection (2) Definition (3) Automation
  • Catalog organization, 3 types of organizations: impact, vulnerability, transformation type

[13] Munawar Hafiz, Paul Adamczyk, and Ralph Johnson. Organizing security patterns. IEEE Software, 24(4):52–60, Jul/Aug 2007.

Notes: Carvalho-abstract - workflows.pdf

A Distributed Reinforcement Learning Approach to Mission Survivability in Tactical MANETs
M Carvalho - csiir.ornl.gov

Found via Gamble. According to Google I've read this paper before, around May 11th. The name Carvalho looks familiar at least, I probably printed this out awhile back.

Gamble's question after reading this paper: How is survivability different than adaptability?

Based on some of the work in the paper, I'd guess that survivability is the ability of a system to perofrm the task that is needed. Adaptability is the ability for the system to change at runtime. Adaptabiltiy is needed for survivability, but it doesn't guarantee it.
  • Definition: Tactical Mobile Adhoc Networks = Tactical Manets
  • "Mission survivability" is different than "System Survivability" which is interesting
  • For the mission to survive a user may not care about the system survivability
  • System failure leads to mission failure
  • Paper is about reinforcement learning, which probably means agents, which probably means its beyond my understanding
  • Uses workflows, ordered set of tasks (could be BPEL?)
  • They ignore complex workflows (loops, parallel paths, etc.). No real justification for this, they just say "for simplicity"
  • Cost based analysis, optimal mapping between tasks and resources
  • "Solutions must be able to dynamically adapt to the overall constraints of the system and mission demands"
  • In this paper: They know the probability of failure of a node, and then want to generate workflows with minimal probability of failure. Their approach: reinforcement learning, could also be done with genetic algorithms I'd think
  • Kind of interesting notification system to indicate a node has a failure in it.

Sunday, May 24, 2009

Tuesday, May 5, 2009

Paper review: " Edge Mashups for Service-Oriented Collaboration"

Found in IEEE Computer May 2009 Issue.

Interesting because they introduce a new phrase similar to what we've been researching. Service Oriented Collaboration (SOC). Addresses the needs for SOCs in health care, disaster response, military operations, and professional dialog/collaboration.

Key goals: Non-programmer should be able to rapidly form a collaboration (workflow?), would like to overlay data from various sources, dynamism at runtime (new data sources, presentation, durable missives, etc.), various network protocols, mobile devices

"Live Objects for SOC" - a model. A lot of their work seems to be targeted towards data issues.

Identifies standardization needs, specifically standardized layering and standardized protocols. "Lacking standards, each protocol is just a black box, and the platform can't determine when one can safely be substituted for another." -- couldn't agree with this more.

Link


Citation: Ken Birman, Jared Cantwell, Daniel Freedman, Qi Huang, Petko Nikolov, Krzysztof Ostrowski, "Edge Mashups for Service-Oriented Collaboration," Computer, vol. 42, no. 5, pp. 90-94, May 2009, doi:10.1109/MC.2009.155


They're from Cornell University.

Paper review: "Daios: Efficient Dynamic Web Service Invocation"

Found in IEEE's Internet Computing May/June 2009 issue.

Motivation: Service unavailability, Discovery of service with higher QoS level
Problem: Binding is handled independently of provider
Existing approaches: Stubs, WSDL
Requirements: Stubless service invocation, protocol independent, message driven, asynchronous communication, acceptable runtime behavior
Approach: Similarity algorithms, Structural distance metric, Abstraction of RPC issues away from invoker, Templates
Also includes: A summary of web service invocation frameworks (Apache WSIF, Apache Axis 2, Apache CXF, JAX-WS)

Keywords: service-oriented architecture, Web services, WSDL, REST, dynamic invocation, Web Services Description Language, Representational State Transfer

Link

Citation: Philipp Leitner, Florian Rosenberg, Schahram Dustdar, "Daios: Efficient Dynamic Web Service Invocation," IEEE Internet Computing, vol. 13, no. 3, pp. 72-80, May/June 2009, doi:10.1109/MIC.2009.57

Vienna University of Technology

Proposal Presentation

Classes are winding down and now I'm back to working on the SPE paper. We've withdrawn our submission about the Delta-Federations from the Congress. Here's a link to my proposal presentation though (Link). Looks like I'll get an A in Computer Security too, which is good. That course has been a significant drain on resources, but not terribly hard to take.