OMA TestFests - Overview Questions

What is an OMA TestFest?

The OMA Interoperability Process (IOP) defines Interoperability Testing Events designed to help achieve a number of goals for OMA and its Members. These goals are:

1. Ensuring the quality of OMA specifications, and
2. Enabling vendors to verify and test the interoperability of their product implementations.

The OMA Interoperability Process (IOP) has been established around the concept of regularly held Testing Events which are commonly known as 'OMA TestFests'. These are hosted by OMA in connection with an operator or vendor, in a confidential and secure testing environment, where member companies can bring their implementation(s) to test in multiple multi-vendor combinations.

OMA staff manage the organization and implementation of an OMA TestFest. This includes securing venues and managing the necessary infrastructure to allow members to successfully test their OMA Enabler Implementations. OMA staff recruit Operator who are Members of OMA who are willing to allow their lab facilities to be used for an OMA TestFest. The staff work directly with the Operator to ensure all elements needed for successful testing are in place.

Where and when are they held?

Roughly every 2-3 months in a variety of locations throughout the world in both Test Labs and suitable hotels with the ability to handle the amount of people and operator support that is required at TestFest.

As a Member of OMA you can check TestFest schedule and you will also receive email notification that registration for the next TestFest is open. The Email notification will direct you to a password protected website (you should have received a password to access the OMA protected website when your company joined OMA). In this area, you will find all the details of the TestFest venue and the TestFest registration page.

If you do not have a Username and Password to the site, please follow the 'Apply for a Login' link at the top of this page.

Engineers from OMA Member companies are invited to participate in the TestFest, as long as they fulfil the following criteria for entry to the event:

• They are at least a Supporter Member of OMA
• They have a unique implementation of an OMA Enabler, developed by the company or in direct association with a third party
• They have implemented all Mandatory features of the Enabler to be tested
• They have passed any entry conformance or other preparatory tests defined for the event, and
• They have signed a Non-Disclosure Agreement (NDA) prior to attending the TestFest.

The following are excluded from participation:

• System Integrators bringing third party or proprietary products for testing/evaluation
• Sales and Marketing Staff
• Observers, Press, Students, and
• Resellers, Distributors and other Dealers

Not more than 4 engineers may accompany any one registered product/implementation unless permission has been sought from OMA staff.

The objectives of the TestFest are:

• To verify the interoperability, stability and quality of the participating company implementations in a multi-vendor environment
• Provide feedback on implementation and operational issues to OMA members developing the specifications, and
• Provide a peer-to-peer networking environment for participants to improve their own implementations and their contributions to the OMA specifications

All OMA TestFests are run as confidential events and therefore all participating companies are be required to agree to sign a Non-Disclosure Agreement (NDA) (please see below), preventing the discussion or distribution of testing information and results beyond the community of participants in the TestFest.

Where a participating company wishes to bring engineers from another company to assist in their testing efforts, please note that it is the responsibility of the participating company to ensure that the 3rd party engineers attending are legal covered and obey the requirements of the OMA NDA. OMA will work with participants to ensure a full level of confidentiality is available to ALL participants, including protecting product identities, designs, product capabilities, test results, etc.

OMA member companies wishing to participate in OMA TestFest must have a Non-Disclosure Agreement (NDA) (these should be dated January 2009 or later) on file with OMA.

Non Disclosure Agreement (Jan 2009)

Companies who do not have a signed Non-Disclosure-Agreement (NDA) on file with OMA prior to the start of the TestFest will not be permitted to participate.

Generally TestFests are designed to be non profit making events. The fees charged are used to cover the cost of the facilities being used by the participants. While some events, when held in an Operator's lab are cheaper, others in hotels are more expensive. Over a number of events, OMA expects the events to break even for the organization. The Registration fees typically include:

• Transport from a selected hotel to the test facility (if appropriate or necessary)
• Rental and setup of the test area
• The provision of power & internet connections (both wired and wireless)
• Test SIMS and their usage costs
• The Organization, scheduling and general management of the event
• Breaks and/or lunch for each participant for each day of the Testing event
• Other services as required

The current standard pricing for participating in an OMA TestFest has been set at:

For Registrations made more than 05 weeks before the TestFest

• $300 US Dollars per Server implementation to be tested at the event
• $300 US Dollars per Smartcard implementation to be tested at the event
• $400 US Dollars per Client implementation to be tested at the event

Registrations made between 05 weeks and 03 weeks before the TestFest:

• $400 US Dollars per Server implementation to be tested at the event
• $400 US Dollars per Smartcard implementation to be tested at the event
• $500 US Dollars per Client implementation to be tested at the event

In addition to the registration fees, there are also participant fees:

• $450 US Dollars per engineer participating if your registration is received more than 05 weeks before the event
• $650 US Dollars per engineer participating if your registration is received by between 05 weeks & 03 weeks before the event

Discounts are available off of these fees for companies who register early, who are testing a Server and who are testing a Server remotely. Full details of these discounts are available on the Registration page for each event.

I have registered, now what?

At the close of registration, OMA have to evaluate whether there is enough clients and servers for each enabler for them to test against each other. If you are the only client or server registered for a particular enabler and you have no one to test against then unfortunately we will have to cancel the testing for that enabler.

Will I still be charged?

Once you have registered, you are not charged straight away. There is a cancellation deadline 02 weeks prior to the start of a TestFest and your credit card is only charged after the deadline and it is confirmed that your enabler is being tested.

Where do I stay?

As part of the organization of the event, OMA will select one or more hotels where participants, in most cases, can take advantage of discounted rates for the event. Where the event takes place in a hotel, then OMA will typically negotiate a room block for participants to offset the cost of the event meeting space and provide a discounted room rate to participants. This keeps the overall cost of the event down for participating companies.

Where an event happens in the lab of an operator or in a separate convention centre, without a hotel attached, then OMA will typically recommend a number of local hotels across a variety of price ranges to allow participants to select the type of accommodation best suited to their budget.

In addition in certain destinations, if OMA select a particular hotel that is more than a few hundred meters from the test location, then a transfer service will also be organized to facilitate transferring to and from the hotel and the test location.

If you have any further questions regarding the logistics of the TestFest, please contact the TestFest logistics team.

OMA has developed a TestFest webpage, which contains Technical information on the enablers being tested:

http://www.openmobilealliance.org/tf/index.asp

In case of incomplete test documentation being available on the OMA Website at the beginning of registration, OMA Staff will contact each Participating Team who registers for that enabler, as soon as the documentation is available, so that the participating team may download the correct documentation from the OMA TestFest website. Test material to be used for each Enabler TestFest can be found under each Enabler specific link on the above reference. All testing at OMA TestFests must be carried out in accordance with the OMA Interoperability and Process documents:

• OMA-ORG-IOP-TestFest_Participation_Guidelines
• OMA-ORG-IOP-Process

In the lead up, you will receive a number of emails with instructions and requirements for attending the TestFest. In addition, all companies participating must submit a completed Enabler Implementation Conformance Statement (commonly known as an EICS) for each implementation they plan to test.

The EICS for each implementation describes the features and functions that the participant company has implemented in the product. It is a requirement that all mandatory functions of an Enabler are implemented by the participant company prior to that implementation being registered for and taking part in a TestFest. In addition to the EICS, some Enablers have a conformance test tools available for them. Where such a Conformance Test Tool exists, companies registering to participate in the event must also submit the appropriate execution report from the Conformance Test Tool to indicate that there product is ready to perform Interoperability Testing at the event.

EICS and Conformance Test Tool execution reports are required to be submitted to OMA's Trusted Zone at least two weeks prior to the start of the TestFest.

The Trusted Zone is a special function of OMA's Testing events that are designed to protect and keep the confidentiality of your companies' Confidential Information of the participants at the TestFest events

In the days before you arrive, you will receive an agenda and instructions on where to go and what time to report for registration on the first day

Your first of point of entry will be the TestFest registration desk to pick up a number of important items such as your badge, SIM cards, network coverage information etc.

Your badge MUST be worn at all times. Other information such as Testing Infrastructure diagrams, lab layouts and generally information relevant to the TestFest are provided, as well as the distribution of IP addresses for testing.

A selection of meals (usually lunch), light snacks and soft drinks are provided for all the TestFest attendees throughout the day and occasionally (depending on the venue) a light breakfast too. On one evening of the Enabler Testing days, we provide a social event usually away from the Test venue, giving you the chance to unwind and meet your fellow engineers in a relaxed and friendly environment. Transportation, meals & drinks are provided and occasionally some entertainment too. A secure area is provided for those who wish to leave bags and/or equipment overnight whilst you attend the social event. Anyone with special dietary requirements is also catered for.

You are given a formal testing schedule so that you have ample time to test against the other clients and servers who are there to test their equipment too. You will be required to complete a Test Session Report (TSR) for each test session. These reports record the results that apply to all of the test cases that are part of the Enabler Test Specification defined by the OMA members. You must record your results as Pass (P), Fail (F), Inconclusive (I) or Not Applicable (NA). The TSR is confidential to the companies participating in that session and is not permitted to be circulated outside of that group of companies without prior written permission of all the other companies which have participated in that test session. At the end of the TestFest, the Trusted Zone collects the reports and produces two summary reports from the TestFest. These reports are a Product Test Report (PTR) and an Enabler Test Report (ETR). A Product Test Report is a summary of all the results achieved by one implementation participating in the OMA TestFest. This report only refers to the number of other implementations that have been tested against. The report information is private and confidential pertaining only to the participating team to which it refers. It is at the discretion of the participating companies to keep this confidential or circulate it to prospective business associates. The Enabler Test Report (ETR) is a summary report of all the PTR's from each TestFest. This report is an indication to the OMA Technical Plenary of both the completeness of the OMA Specification, as well as the quality of the implementations that are being tested. The ETR is available to all members of OMA.

One of the important aspects of any interoperability testing session is to record and process the problems found in the implementations being tested. Some problems are purely a company's implementation issues, but many problems related to errors in the specification, and also to areas of different interpretation between implementations. These issues all need to be recorded by the participants at a TestFest and discussed by the relevant working groups to determine how a problem should be resolved or an area of misinterpretation tightened up to what the working group actually intended to begin with.

OMA makes a Problem Reporting tool available to all TestFest participants that allow the easy reporting of problems to the Interoperability Working Group and to the Specification Working Groups.

The OMA Problem Reporting System can be found at: http://www.openmobilealliance.org/TestFests/Problem_Reporting.aspx

Documentation required to attend an OMA TestFest

All companies participating in an OMA TestFest MUST submit, to the OMA Trusted Zone*, a completed Enabler Implementation Conformance Statement (commonly known as an EICS) for each implementation they plan to test at the event. The EICS's for each implementation describes the features and functions that the participant company has implemented in the product. It is a requirement that all mandatory functions of an Enabler are implemented by the participant company prior to that implementation being registered for and taking part in a TestFest.

No later than 2 weeks before the event, all Participating teams MUST submit a completed EICS for their enabler to the OMA Trusted Zone.

* The OMA Trusted Zone is a special function of OMA's Testing events that are designed to protect and keep confidential the Company Confidential Information of the participants at the TestFest events.

Documentation provided during an OMA TestFest

While participating in the test sessions at OMA TestFests, participants will be required to complete a Test Session Report (TSR) for each test session. These reports record the results that apply to all of the test cases that are part of the Enabler Test Specification defined by the members of OMA. Results will be recorded as 'Pass' (P), 'Fail' (F), 'Inconclusive' (I) and 'Not Applicable' (NA).

The TSR is confidential information of the companies participating in that session, and is not permitted to be circulated outside of the group of companies without prior written permission by all other companies which have participated in that test session.

Further Information

We hope the above has answered any questions you had about attending an OMA TestFest. For additional information or questions about OMA TestFests, please contact the OMA staff