Automated testing, monitoring and load testing for EPIC

Getting ready for an Epic implementation or an upgrade? From facilitating physician access to a patient chart or lab results, data mining for clinical research, integrated claim processing or patient engagement, the benefits that an EHR system like Epic will provide to your organization are many. Yet it is one of the biggest and most complex projects you will ever embark on as a healthcare IT professional. For example, a recent $55million Epic implementation at MidMichigan Health required 160 MidMichigan and Epic staffers working around the clock for over a year. Therefore, considering project complexity, cost and length you should be primarily concerned about on-time and under-budget completion as the primary metrics for implementation success, right? Wrong!


While on time delivery, stability and uptime should be critical considerations, your success will be ultimately measured by adoption and user satisfaction. However, there is plenty of room for IT improvement in this front. Consider that only 34% of doctors were satisfied with their EHR systems in 2014 (American Academy of Family Physicians), and doctors and nurses across the board continue to demand an extreme EHR makeover. Want to avoid chaos and frustration and boost user satisfaction? Plan, test, load test and monitor your EHR systems from a user perspective, for all your users, for all roles. Here are some quick pointers to help you get started.

1) Plan– Take the time to properly define record fields to ensure consistency across all facilities, and discuss common workflows and flows with your users (doctors, nurses, financial personnel, administrative staff, patients…). What functions are used most widely? What is the most frequently accessed information?

2) Map and optimize business processes- Your users want to spend less time interacting with the software and more time elsewhere. Therefore, map and document all user requirements and functions, for each group of users. Could you reduce the number of clicks? Could you improve searching? What is a user more likely to want to do next? What are must-have customizations? Consider how to optimize common workflows into new workflow models.

3) Test, test and re-test again (automate your testing!) All functions and critical end-to-end business processes should be thoroughly tested and validated. Begin functional testing as early as possible, so users can provide feedback on common processes and validate reporting structures. Focus on performance testing as well to oversee application response time, a common end-user frustration pain point. As your implementation progresses, automate your testing to save time and easily verify updates, and thoroughly test and re-test as needed to ensure top quality and performance. Look for a test automation solution where you can easily replicate and automate complete end-to-end user interactions with Epic in the form of testing scripts (e.g. logging , search for a patient record, access latest lab results, add visit information ), which can be later on re-used for load testing and monitoring as well.

4) Load Test. As more users place additional demand on your Epic system, performance could slow down and even bring operations to a halt. Ensure readiness for traffic spikes and demand fluctuations with load testing so you can verify that all critical functions and workflows will work well under peak conditions. Build comprehensive test plans that include realistic scenarios with various load levels and ramp up times, and multiple users concurrently performing different actions. In addition, since Citrix is a trusted solution provider to more than 90% of the largest healthcare organizations make sure that you will be able to replicate and measure REAL user activity using the ACTUAL client connection GUI for each load test. That way the most important metric is measured and analyzed – the end-to-end client side response time, so you can verify that SLAs will be met for all pre-defined levels of load.

5) Proactively monitor your Epic system. Simulate and measure availability and response time for all critical Epic business processes you have documented and tested on a 24x7x365 basis, taking response time measurements end-to-end and for each step. Ideally you should reuse testing scripts for proactive SLA monitoring, so you get alerted at the first sign of trouble, when Epic starts to slow down, before your users are impacted.

6) Continue to listen to your users. Stay aligned with your users’ needs and look for ways to continuously boost user satisfaction. Consider re-defining workflows, planning additional customizations, allocating more hardware resources or organizing more Epic training, and keep all lines of communication with users open by hosting regular meetings. Additionally stay on top of key industry trends (e.g. shortcut key strokes could prove deadly with health records and Epic warns providers who use copy and paste) and periodically review testing and monitoring reports to identify the need for Epic performance optimization.

Want to learn more? Check out CitraTest (test automation) CitraTest VU (load testing) and CitraTest APM (proactive response time monitoring from a user perspective).

Good luck with your Epic implementation!

End-to-End Monitoring for Cloud environments (XenApp / XenDesktop / RDS)

You rely on XenApp, XenDesktop or Microsoft RDS to deliver Windows, Linux, web, and SaaS applications or desktops to your employees and customers on any device, anywhere. To meet changing business needs and quickly scale, most IT organizations are now supporting applications on-premise, in the cloud or in hybrid environments. In fact, 76% of workloads are expected to run in the cloud within two years (451 Research), so a growing number of organizations are moving XenApp and XenDesktop to Azure or Amazon WS or Microsoft RDS to the Azure cloud. Are you ready to continue to deliver optimal user experiences, regardless of your deployment model?

Cloud Monitoring

In a culture of fierce competition and high expectations, end-user experience rules. Users expect – and demand – that the performance of their virtual applications and desktops match or exceed the performance of their physical applications and desktops. Yet, delivering virtualized applications or desktops from the cloud adds a new layer of complexity to troubleshoot the root cause of performance issues. Not only are you dealing with complex virtualization technology with multi-tier components affecting user experience ( e.g. hypervisor, delivery controller, provisioning mechanism, XenApp and/or XenDesktop VDAs, Citrix database, file servers, databases…), but you also rely on a cloud platform that can introduce performance degradations. And even a slight glitch can result in user disconnects, slow logons, keystroke lags, or screen freezes—all extremely frustrating for your users. To top it all off, unlike an underperforming physical desktop frustrating a single user, a virtualization performance issue affects many users.

Here are the 5 pillars to consider as you build an optimal End-to-End Cloud Monitoring strategy:
• Since you are streaming applications within a Citrix or RDS-based environment running on premise or from the cloud, you need a testing and monitoring approach based on image recognition (object recognition will not work). All user action should be automated, tested and measured, so look for a solution that lets you capture, store and edit baseline bitmap images to define your test and monitoring scripts.
• With so many potential failure points, it is time to revamp quality and test automation. Functional testing, smoke testing and performance and load testing should be continuously embedded in your deployment cycles. As users place added demand into your Citrix or RDS environments you need to ensure readiness to handle it, regardless of your deployment model.
Test, benchmark and compare. When migrating technology to the cloud or rolling out a new application, a good approach is to load test and baseline prior to roll-out, then load test post-deployment and compare (to ensure no performance degradation for your users). Look for the ability to reuse testing scripts for production monitoring to ensure cross-functional alignment.
Proactively test and monitor end-to-end from your user perspective . Simulate and measure availability and response time of critical end-to-end user transactions on a 24x7x365 basis, to find problems before your users are impacted. Look for proactive user-centric monitoring that periodically executes synthetic transactions, taking user SLAs response time measurements along the way to ensure that your application are working effectively, alerting you at the first sign of trouble.
Focus on performance analysis. Performance optimization boosts external-facing application speed and employee productivity and drives more revenue. Therefore you should benchmark and analyze application performance on an on-going basis to identify areas for improvement.

Are you ready to take your end-to-end monitoring for cloud environments (XenApp, XenDesktop, RDS) to the next level? Check out CitraTest, CitraTest APM and CitraTest VU. All you need is accessibility from a Windows client to start testing and monitoring your applications, including cloud-based applications!

Did you know that Tevron its celebrating 16th year in February 2017? Our first Citrix automation and performance measurement solution was announced back in 2011! Learn more about who is using Tevron at

SLA Monitoring tips for IT Pros

Whether you like it or not, user experience rules!
In fact, your end users have the ultimate say on whether they are satisfied with the speed, availability and performance of your applications. So a traditional infrastructure-monitoring approach is no longer enough for IT. Moreover, the only way to ensure business success is to build an IT strategy centered on SLA Monitoring from an end-user perspective, so check out these tips:

1) Define your SLA monitoring plans cross-functionally. Building a killer SLA monitoring strategy requires cross-functional collaboration, proper planning and a dose of diplomacy to ensure alignement. For example, are you changing infrastructure providers or moving to the cloud? Are you rolling out some new applications in the future? Are you updating your Website or embedding new third-party components like Web tracking analytics or Social Media widgets? Get all stakeholders together (Development, QA, IT, business owners and marketing teams) to get buy-in and consensus on your SLA monitoring plans at a high level from the get-go.

2) Monitor your SLAs from a user perspective. Your applications fuel your business, so they must consistently perform well, and ultra-fast. Continuously monitor end-to-end transactions from a user perspective, proactively taking response time measurements to ensure user SLAs are met. Check out CitraTest APM! By automating the driving of ANY application — just like a real user would do — visually examining the desktop and responding to changes, driving the keyboard and mouse, waiting for the next step and measuring response time– CitraTest APM ensures that all critical aspects of your applications are available and working effectively, alerting you in real-time at the first sign of trouble – before your users complain.
3) Monitor critical transactions first. Take the time to map out critical end-to-end flows in your applications. For example: New application code, marketing registration forms, shopping carts, Salesforce record retrieval, key HR functions – they should all be monitored. Focus first on those transactions that drive revenue and fuel your business operations and start monitoring them on a 24x7x365 basis to ensure they are performing top-notch.

4) Oversee SLA response time historically and in real-time. Build comprehensive monitoring scenarios with custom grouping based on physical locations (Headquarters, NYC, remote offices) or monitoring functionality (Application A- SAP, Application B- Salesforce, Application C- Your Main Website, etc.). Everything should be measured in absolute values, percentages or statistical deviations. For example, SLAs for application load time, login time, a Hadoop transaction execution, how long it takes after a “button” is clicked or until the next window or item appears, and so on. That way, you can understand how each application performs per location, per step or per group, both historically and in real-time, identify underperforming components or problematic trends, and prove that SLAs are being met.

5) Look for drill-down capabilities to simplify triage and troubleshooting. CitraTest APM ‘sees’ exactly what the real user experience is, so every monitoring script is built with simple, easy-to-understand visual references, and are easy to create and maintain. When a monitoring script fails, CitraTest APM will take a screenshot to help you analyze the root cause. You can even click and analyze recent incident grids – up to 150 trace logs and screenshots — or visualize script fail logs in sequence to simplify troubleshooting tasks.

6) Don’t forget your mobile users! Smart devices are not only competing for PCs’ place in your users’ lives, or in the enterprise, – they are replacing the experience. In fact, smartphones and tablets now account for 60% of total Internet time (Source: comScore). And mobile user expectations are on par with, if not higher than, their desktop counterparts. Therefore, just like you monitor SLAs and response time for your desktop users, look for SLA monitoring for both mobile and desktop users. Check out CitraTest Mobile

7) Focus on application quality. Before you even define your SLA monitoring plans, think about testing your applications from a user perspective, and test real end-user experience in pre-production! With CitraTest, you can generate real mouse activity and keyboard strokes and then test, measure and validate the images on the screen — just like a real user — and easily reuse those scripts for ongoing SLA monitoring in production via CitraTest APM.

Are you ready to take your IT and SLA monitoring to the next level?
Check out CitraTest, CitraTest APM and CitraTest Mobile

Good luck with your SLA monitoring!

Monitoring Cloud-based Applications Best Practices

What is your cloud monitoring strategy? After all, a whopping 93% of organizations in 2015 are running applications in the cloud or experimenting with infrastructure-as-a-service, while 82% of enterprises have a hybrid cloud strategy, up from 74% in 2014 (Source: RightScale 2015 survey).
Cloud pic

As you grow, and join the ranks of these enterprises moving to the cloud, or leveraging cloud-based applications, there is one thing that you should not forget. Regardless of where your applications are hosted, your users will hold IT responsible for their user experience! To ensure smooth operations and user satisfaction, you need to monitor cloud-based applications from a user perspective, just like any other application in your portfolio. Here are some quick tips to jump start your cloud monitoring efforts.
1. Prioritize your cloud-based applications. With an increasing number of employees bypassing IT and going rogue to the cloud, it is the wild, wild, west out there.  From accounting, invoicing, HR, CRM, email or collaboration tools, the list of applications your employees rely on is long! Since you can’t monitor it all at once, identify those critical applications that must perform well in order to run your business (e.g. internal applications migrated to the cloud, cloud-based CRM, ERP, EHR systems), and monitor these cloud-based applications first.

2. Identify critical transactions to monitor. Put on your “think from a user perspective hat” and map out common functions used by your power users (those driving the most productivity or revenue). Or better yet, schedule a meeting with business stakeholders to identify critical functionality from their perspective. For example, if you are monitoring a healthcare portal, outline critical transactions to monitor such as:

  • For a doctor- Opening a patient record, logging recent activity, updating prescription details
  • For a patient- Logging in, reading a doctor email, searching for care

All of these functions should be proactively monitored on a 24x7x365 basis (check out CitraTest APM!) to identify early problematic trends before your users are impacted.  After all, a cloud migration project introduces risks of performance degradation (e.g. noisy cloud neighbors in a multi-tenancy deployment resulting on “stolen CPU”, or processes that are not written to effectively use memory, which could overload the allocated memory in an Amazon EC2 instance).

With CitraTest APM you can continuously execute synthetic transactions, taking response time measurements along the way to quickly identify problematic trends. By automating the driving of ANY application — just like a real user – and visually examining the desktop and responding to changes, typing on a keyboard or on a mobile device or driving the mouse, CitraTest APM ensures that all critical aspects of an application are available and working effectively, alerting you in real-time at the first sign of trouble.
3. Decide polling frequencies and alerting policies. A good rule of thumb is to monitor key transactions more frequently (e.g. being able to send a sales proposal is more critical than reporting on sales pipeline) to identify performance degradation signs earlier, and define who should be alerted in the event of warning/critical threshold violations. With CitraTest APM, you can configure the number of response time violations that will trigger an alert to eliminate false positives, easily schedule monitoring tests or disable alerting on scheduled maintenance periods. You are in control of your monitoring!

4. Define custom reports Since different metrics are important for different stakeholders, take the time to map out role-based reports with custom information for each team (per application, per transaction, per functionality, etc.), and automatically distribute reports on an on-going basis (daily, weekly or monthly basis) to keep everybody informed and aligned.

5. Centralize IT response procedures and workflow. Don’t treat cloud-based application monitoring as a separate beast! After all, you are responsible for user experience for all applications hosted on your datacenter or in the cloud, so you want to add cloud monitoring to your existing IT monitoring framework to increase efficiency. Plus you want to use the same APM monitoring solution for all the applications you own and/or support.

Are you ready to boost performance for your cloud-based applications and find performance problems before your users do?

Check out CitraTest APM and CitraTest Mobile

Good luck with monitoring cloud-based applications!

Test Automation for Citrix, Microsoft Terminal Services or RDS Deployments

Assuring application quality and customer satisfaction is the end goal of every organization. However, achieving it in a Citrix, Microsoft Terminal Services or RDS deployments is a whole different ball game –no underlying objects or controls are delivered to the client, instead an image of the application is delivered to the client UI. Therefore, software testing tools built to “see” Windows objects that require specialized knowledge to understand and a myriad of different techniques to manipulate in tests simply will not work, leaving you in the dark, and leading to underperforming applications and unsatisfied customers.


There’s a better solution, and luckily for you it’s super easy to use. If you are looking to automate functional, regression or performance testing in a MTS, Citrix or RDS environment, you will need a test automation solution designed to rely on advanced intelligent image recognition capabilities – automatically built into every testing script – to simplify testing configuration, and reduce your time and efforts. And this is exactly what CitraTest VU delivers.

Let’s illustrate further with Tevron’s real-world customer stories and ROI from the trenches:

Story 1:
Boost application quality within an RDS-based environment

“Since we stream our customer-facing applications within an RDS-based environment, none of the tools out there would work within our environment, we couldn’t test at all since we couldn’t recognize Window objects, the basis for other test automation solutions.”

The result? “The first time we used CitraTest VU load testing tool we identified that our Silverlight application choked with 100 virtual users (VUs), which was an eye opener for us. We wanted to test with 200 VUs, but we couldn’t finish due to a major memory hog. Now we regularly share all CitraTest VU results with Development, QA managers and business analysts, and we jointly analyze this information to fine tune our applications, change configuration settings, optimize performance and gain peace of mind.”

Story 2:
Test Automation in an Agile world

“Now that we are an Agile-based development tool, development cycles are much shorter, and we have much less time to test. Speed is a must, we need results fast, and we need to be able to quickly tweak our test scripts and react to frequent application functionality changes.”

The result? “CitraTest VU is reliable and easy to use. Scripting is fairly easy, I pretty much learned it on my own, and I was up and building scripts in no-time. Plus I can test other areas while my scripts are running, so application quality is much better. Now we use CitraTest to load test every time we roll out a new application (every 3 to 5 months). We also use it for regression testing, plus we use it for smoke testing every day on our builds. This has saved us considerable time!”

Wouldn’t you want to free up your time to do other essential testing? Who wouldn’t!

Therefore, look for a non-intrusive test automation solution that uses the same elements as a user when interacting with an application (visually examining the desktop and responding to changes, driving the keyboard and the mouse) and embeds image recognition technology to replicate, test and measure any real world transaction using the actual client interface. That way you can capture and compare screen images to a user-created baseline, the basis of Citrix, Microsoft Terminal Services and RDS deployments, and quickly identify slow components, functional defects and other quality problems that need to be addressed before production release.

Are you ready to learn more? Check out CitraTest (test automation) and CitraTest VU (load testing for Citrix and Microsoft Terminal Services environments) from Tevron.

Good luck with your quality endeavors!

Working on migration projects or DB rolls-outs? Simplify your life with automation!

Infrastructure roll-outs, cloud migration and new Websites top the list of this year’s IT projects. And let’s face it – even when everything goes according to plan (as if that’s ever the case!), these endeavors take up way too much of your precious time, keeping you from finishing other tasks, and keeping you for long hours in the office. The solution is simple – it’s time to streamline and AUTOMATE!


Think about automation as a series of steps that you can automatically generate and drive from a computer in sequence using a “virtual user” (copy, paste, type, click, generate a record, etc.), until a job is completed, WITHOUT any human intervention (other than building the automation script or a set of instructions that control virtual users behavior). For example, manual entry or database rolls-out are great examples of time-consuming, tedious and prone-to-errors tasks that you should automate to reduce costs, save time and accelerate project completion.

Let’s illustrate the point with Tevron’s real-world customer stories and ROI from the trenches:

Story 1:
Automate DB roll-outs (60,000 records in 4 days!

“I walked into a room and I saw 6 people in a room moving 60,000 records manually, countless weeks ahead them”. So I told them “Wait. I work in QA, I can build a CitraTest script and automate this task for you”.
The result? Instead of using a team of 6 working on this phase for months, the job was completed by CitraTest in only 4 days.
The pay-off? Smiles all around. I was able to save MANY days of work and effort, drastically cutting costs and accelerating time to completion

Story 2:
Automate Data Entry

“We were converting a legacy system into a new system and we found out too late that our vendor didn’t provide any tool to automate data migration. The team was desperate, many days, many man hours ahead of them to manually tackle this phase. CitraTest automatically completed the process and saved us a lot of time and efforts. Besides scripting anything (so you can automate all manual tasks), it has a nice recovery feature so you can re-start your data entry at any point without losing any data“.
The result? Peace of mind and complete accuracy, plus big savings in time and cost

If you are new to automation, speak with your QA/Testing counterparts to discuss how to go about building your automation scripts. For example CitraTest is a software test automation solution that can be used for both QA pre-production testing (functional, performance, regression testing, load testing) and to automate complex, tedious and time-consuming manual IT tasks. You can script against any application (Citrix, Terminal Emulation mode, legacy applications, all Web browsers, all Thick Client apps, etc.) that is accessible from a Windows client. CitraTest automates manual tasks by remaining a back-ground operation that generates real mouse activity and keyboard strokes and then responds to the images on the screen – just like a real user. Every user action can be replicated and automated.

Ready to learn more? Check out CitraTest!

The next time you walk into a room and see your peers engaging in manual tasks (or you are in the mist of completing tedious and time-consuming manual tasks), think about automation to get the job done, faster, error-free and much more cost-effectively. Great for both QA and IT teams.

Good luck!

Load Testing for IT Pros

If you think that load testing (or the act of stress testing your applications to ensure quality user experiences under heavy traffic) is only for your QA counterparts, think again. In 2013 a whopping 23% of IT budgets were allocated to application quality according to Capgemini’s World Quality 2013-2014 report, up from 18% in 2012. When you think about it, it makes sense. After all IT is the first line of response when users complain and application issues arise.
Blog #3

While load testing can be intimidating for IT pros at first, it shouldn’t be when you are armed with the right testing solution. Wouldn’t you prefer to triage and resolve overloaded Citrix/MTS issues early on, before large volumes of users are impacted? That is exactly what you can do with load testing, plus you can protect your organization of hefty compliance violation penalties and security breaches. For example in the summer of 2012, the University of Chicago Medical Center had an issue where overloaded Citrix servers were logging users under wrong profiles!

Here are some quick tips to help you get started:

1. Look for a non-intrusive performance testing solution to avoid performance impact in your production environments. Ideally, “load generating machines” should be used to generate user load against the server environment under test, while “measuring machines” should measure response times and gather server performance metrics on the back-end.

2. Select a tool that doesn’t require advanced programming skills. Unless you want to show off your programming skills, or move full time to development, look for a solution that offers an easy to use point-and-click scripting wizard, so you can easily build your test scripts.

3. Make sure that you will be able to replicate and measure REAL user activity using the ACTUAL client connection GUI for each load test. That way the most important metric is measured and analyzed – the end-to-end client side response time, so you can verify that SLAs are met for all pre-defined levels of load, plus is the most accurate way to identify underperforming application components at peak usage time. For example, each CitraTest VU test script executes on its own “desktop”, and opens its own client connection to the server-under-test, just as a group of real users would.

4. Double-check that you can create and execute large volumes of real user transactions against your production environment. That way you will be able to select the right level of load (or virtual users) to fit your custom needs. Since number of employees, locations, offices, peak requests, peak times, and so on will change over time, you need to have flexibility in your virtual user levels.

5. Build realistic and comprehensive testing scenarios. Consider areas such as key applications to load test, key multi-step user interactions to verify under load (e.g. logging, check a balance, add customer call log, etc.) , numbers of virtual users to test with, as well as ramp-up time. Don’t forget to generate realistic load modeling scenarios with varying number of virtual users for complete test coverage. In addition, as your real users will not all perform the same actions, you need to simulate multiple users concurrently performing different actions in order to realistically stress test your applications, and verify all critical functions and end-to-end transactions are working well under load.

6. Oversee both server performance and end-to-end response time under load. That way you can isolate hardware constraints and bottlenecks, and ensure sufficient back-end infrastructure capacity to handle all user requests and deliver quality user experiences at peak times.

7. Keep all stakeholders in the know. In a world where IT is becoming metrics-driven and ensuring alignment with your business counterparts is a must for success, you need to be armed with the right information. Share your load testing reports (test scenarios, results, virtual users, SLAs, etc.), and keep peers, management and stakeholders in the know.

8. Analyze results. Load testing gives you actionable insight to assess the need to tackle key projects for the business (e.g. capacity planning, new vendor selection, Web performance optimization, etc.). Get together with team members and stakeholders and periodically review load testing results to make better informed decisions.

9. Decide on-going load testing frequency to ensure IT readiness on a 24x7x365. Load Testing is not a “one-time-only” activity. In fact, you should think about load testing whenever you encounter any of these situations outlined below.
o Mounting user escalations (Increased number of open helpdesk issues, escalations and user complaints, intermittent application problems that you cannot reproduce or troubleshoot…)
o New application roll-outs (e.g. new application deployment, bug fixes, new patches or service releases)
o Organizational changes (e.g. mergers and acquisitions resulting in additional employees, offices, branches or locations, business development activities, geographical expansion)
o Infrastructure changes (e.g. datacenter consolidation, virtualization, new server deployments, cloud migration)

10. Check-out Tevron’s CitraTest VU to test Citrix/RDS performance under multiple load levels, and ensure quality experiences, for all users, at all times – under all load conditions.

Did you know that even Citrix relies on CitraTest VU to run their own load testing initiatives when a new version is released?

Your network is a living entity where the number of users, remote branches and locations, business requirements, load levels and users’ expectations change at lightning speed. So load test your applications (external facing and internal apps) to ensure IT readiness for whatever changes come your way! Need more convincing? Check-out how load testing helps you mitigate risks!

Good luck with your load testing efforts!

Stem the tide of user dissatisfaction with IT via application monitoring

You may not have noticed any changes in your organization yet, but it seems that dissatisfaction with IT is on the rise.  To give you some quick data points, 94% of IT projects are unsuccessful , a mere 15% of employees say that they are completely satisfied with their technology management department’s understanding of what they need to be successful, and only 13% of executives are satisfied that their IT organizations are very effective at introducing new technologies faster than competitors.


As your role changes from tech gatekeeper to tech enabler, it is time to address the #1 customer complaint have about IT: “The application is not fast enough!”. Here are some quick tips to help you get the work done.

Tip #1: Monitor your applications from your end-user perspective.
Your applications are the lifeblood of your business, so they must consistency perform well, and perform fast. Otherwise, revenue and productivity will be lost, business operations will be disrupted, and user complains and escalations will go through the roof. An infrastructure-centric monitoring approach is not enough. Instead, you need to simulate and measure availability and response time of critical transactions on a 24x7x365 basis, from your user perspective, as shown below:

• User 1- CRM logging –> Open a customer record –>Add a sales log –>Log-out
• User 2- CRM logging –>Access sales pipeline –>Generate a report –> Log-out
• And so on

With proactive user-centric monitoring you can isolate problematic trends much earlier, and find and resolve issues before your users are impacted.

Tip #2: Proactively monitor all common workflows .
Take the time to map-out who your users are, and how they interact with your applications so you can monitor key user interactions. For example, with Epic EMR, both patients & doctors access MyChart records, so you should monitor Epic EMR response time from both patient and doctor perspectives, as shown below.

Doctor– MyChart log-in –> Access a patient record–> Add visit log–> Save–> Logout
Patient– MyChart log-in –> Access my record–> Search doctor phone information–> Logout

Also, if you are implementing or maintaining SAP, check out some additional SAP monitoring tips.

Tip #3: Find geographical response time discrepancies early on.
In some cases, employees in remote offices could experience slower response times than employees accessing your applications from headquarters. The faster way to find and troubleshoot problems like this is to monitor and compare availability and response time of your applications across multiple locations (Headquarters, Boston, NYC, remote office locations, etc.).

Tip #4: Validate and share SLAs reports with users and stakeholders.
In a new era where customer satisfaction rules, you need to continuously validate and demonstrate your SLAs. Therefore, you should track, report and periodically share your SLAs with management and stakeholders. With a metric-driven strategy in place you can keep stakeholders in the know, and quickly ascertain if you need to focus on app performance optimization or not.

Tip #5: Boost IT efficiency with a single solution to manage your entire application portfolio.
From legacy applications, to client server applications, to web applications, to home-grown custom applications, cloud applications or green screen apps, most large enterprises have a complex portfolio with 250-500 applications to support. The cost of purchasing, configuring and maintaining several monitoring products to support individual applications is too high. Plus lack of integration across monitoring consoles hinders troubleshooting and increases Mean-Time-To-Resolution (MTTR). Instead, look for one solution that lets you test and monitor all applications, regardless of where they are deployed.

Are you ready to boost user satisfaction with IT? Check out CitraTest APM!
CitraTest APM periodically executes synthetic transactions, taking response time measurements along the way. By automating the driving of ANY application — just like a real user would do — visually examining the desktop and responding to changes, driving the keyboard and mouse– CitraTest APM ensures that all critical aspects of an application or service are available and working effectively, alerting you in real-time at the first sign of trouble – before your users complain.

Good luck!

SAP Monitoring DO’s and DONT’s

As an IT pro supporting SAP, or tackling an SAP implementation, simplicity is certainly not the first attribute that comes to mind. In fact, as an enterprise suite of ERP, CRM, PLM, SCM and SRM applications, SAP links and integrates every business process across your organization. While 253,500 customers in 188 countries successfully rely on SAP to accelerate their business operations, inadequate planning, lack of testing during each implementation phase, improper SAP monitoring, or even poorly managed user training can all result in costly pitfalls and mistakes for you and your organization.

For example, a recent $50 million SAP implementation by San Diego — the biggest IT project ever undertaken by the city, designed to provide a core IT architecture for 15 years — ended up tripling employees’ workloads for certain types of tasks! Pair this up with other highly visible SAP implementation failures at Marin County, or at Hershey’s where order fulfillment time doubled to 12 days and you will get the point. Testing and validating SAP functionality and workflows during each implementation phase, and proactively monitoring availability and response time of SAP in production systems, as perceived by your end-users, are critical to your success. Here are some tips to help you get started.


DO #1. Document existing business processes: Since SAP touches so many business functions, take the time to understand and document how each business user (project managers, HR, sales, financial teams, etc.) completes existing processes and common workflows.
SAP Workflow

DO #2. Redesign business processes. Many of the SAP implementation pitfalls we just mentioned were primarily due to simply replicating existing processes instead of redesigning them with SAP for more efficiency. Therefore, don’t skip this crucial, yet time-consuming step, and map and optimize all user requirements, for each group of users, into new workflow models. This information will also help you guide your monitoring efforts later on

DO #3: Reduce the number of customized functions. Try to use as many objects and functions as you can from the off-the-shelf version of SAP instead of writing your own custom code. You might run into quality issues with custom objects later on, plus they might not work as expected if/when you roll-out a new version of SAP.

DO #4: Test and validate each SAP implementation phase – from an end-user perspective. Begin manual testing as early as possible, so users can provide feedback on SAP business processes and validate reporting structures. Later on, automate your testing processes to save time, increase efficiency and thoroughly test and re-test as needed to ensure quality and top-notch SAP performance. Look for a testing tool where you can easily replicate and automate end-to-end user interactions with SAP in the form of testing scripts (e.g. enter an order, generate a shipping receipt, create an invoice) , which can be later on re-used for SAP production monitoring as well.

DO #5: Train your users. Tailor your training accordingly, and plan for role-based training customized to each set of users’ unique requirements


Did you know that the #1 customer complaint faced by organizations that use SAP is “SAP is too slow”? Therefore, instead of infrastructure-centric SAP monitoring, look for synthetic monitoring capabilities to automate the driving of your SAP applications — just like a real user would do — visually examining the desktop and responding to changes, driving the keyboard and mouse and continuously taking response time measurements along the way, on a 24x7x365 basis, so you get alerted at the first sign of trouble, before your real user are impacted. Here are some additional tips for you

DO #6. Build a comprehensive Monitoring Plan. Just like you documented your business processes and workflow for each set of business users, take the time to plan and document your SAP monitoring strategy, including key areas like: Monitoring scripts (so you can proactively measure complete end-to-end user interactions with SAP), Alerting Policies, Notification Groups, Reporting Distribution, Monitoring Locations, Service Level Agreements and so on

DO #7. Focus on Essentials workflows first. Tackle the basics, including key financials, HR/payroll, supply chain processes and workflows, and add more monitoring scripts later. In addition, consider using a dedicated “SAP user” monitoring account to keep your SAP data secure.

DO #8. Look for non-invasive agentless SAP monitoring that will not change your application code. That way you can ensure no impact on your SAP production system, and complete monitoring accuracy

DO #9. Be aggressive in your SAP monitoring thresholds (availability and response time). Since proactive SAP monitoring helps you identify problematic trends early on, before your end-users are impacted, you should baseline and setup both warning and critical thresholds for all monitored SAP transactions. Consider as well monitoring SAP transactions across multiple locations so you can easily identify response time discrepancies across users and locations.

DO #10. Don’t monitor SAP in isolation. Integrate SAP monitoring, alerting and performance dashboard views within your existing IT management tools and frameworks tools to increase IT efficiency and streamline your response procedures

Are you ready to avoid SAP problems and ensure success?

Check out CitraTest (SAP test automation) and CitraTest APM (proactive SAP response time monitoring from a user perspective).

Good luck with your SAP efforts!

IT Best Practices: Monitoring cloud deployments from your users’ perspective

Enterprise cloud spending is expected to triple by 2017, and reach $235.1 billion (Source: IHS), so if you are not already there, it is very likely that you soon will be moving to the cloud.

Today IT success is measured on the value you deliver to the business, and how well you service the end-users you support. Consequently, regardless where your applications are deployed, on premise or the cloud –outside your control–, IT is ultimately responsible for end-user experience. Do you want to successfully move to the cloud and avoid business disruptions and end-user escalations and complaints?

Here are some quick guidelines to help ensure cloud migration success for enterprise IT teams:
IT cloud

1. Properly evaluate your cloud provider. Did you know that Gartner is predicting that 1 in 4 cloud providers will gone under by 2015? Therefore, think long-term, do your prep work carefully, don’t skimp time on your selection process, and closely evaluate key areas like:

  • Current/future cloud locations: It is best to span your cloud architecture across multiple locations to mitigate outage risks. In addition, if your organization operates in Europe, be aware that the EU (European Union) Data Protection Act mandates keeping personal information within the EU, so you need a cloud vendor with global coverage.
  • Data format considerations: Think about in which format your data will be stored, and more importantly, prevention is the best cure, so plan to keep a local copy, or at least instant data access, in case your cloud provider goes under.
  • Business Continuity/Disaster Recovery Plans. For example, evaluate automatic back-up capabilities, and consider an architecture that lets you store back-up data and applications locally, and across cloud locations for added protection.
  • Security. This is a critical area, especially if you are planning to move to the cloud sensitive application data that fall under regulatory compliance (e.g. HR records, patient information, credit card data, financial records, etc.). Therefore, carefully assess providers’ security mechanisms (e.g. Intrusion detection and prevention systems, spam and virus filters, role-based access, SSL tunneling, etc.), examine independent 3rd party audit reports, and make sure you understand who is ultimately responsible for the security of your data.

2. Consider a phased-approach — don’t move all applications at once! Less is more. Start small, moving less critical applications first to make sure that you get it right, and everything is working as expected. In addition plan to test and measure application performance, from your users’ perspective, BEFORE you move to the cloud to baseline your performance, and AFTER your migration is completed. That way you can compare performance pre and post-cloud deployments, and take corrective actions as needed.

3. Monitor application performance, from your end-user perspective, from the get go. Since application content will traverse a complex path of interrelated entities inside and outside your control zone – datacenters, private and public clouds, remote sites and locations, 3rd-party services and components – there will be more potential bottlenecks and failure points that can impact end-to-end application performance, as perceived by your users. Consequently, proactively monitor your applications from your users’ perspective to find problems before they do.

4. Don’t assume that your cloud vendor will take care of overseeing application availability and performance. While your business colleagues may think they can buy cloud services and instantly meet all their requirements, you know better. Your operational requirements won’t change, regardless of where your applications are deployed. If the cloud provider doesn’t fully meet your requirements, you have to fill the gap, so proactively monitor your SLAs to ensure quality and predictability, for all users at all times.

5. Keep all stakeholders in the know. Look for an application performance solution that gives you custom reports and automatic email distribution capabilities (daily, weekly, monthly, etc.) by application, by transaction step, by location, a summary of SLA violations, etc. That way you can easily access and share information across cross-functional teams and stakeholders and keep everybody aligned, a must for cloud deployments.

Are you ready to ensure cloud migration success?

If you haven’t done so yet, check out Tevron’s CitraTest APM, an enterprise-class user centric monitoring solution for ALL your applications, regardless of where they are deployed: datacenters, private and public clouds. By automating the driving of ANY application — just like a real user would do — visually examining the desktop and responding to changes, driving the keyboard and mouse and taking response time measurements along the way, CitraTest APM ensures that all critical aspects of a cloud-based or on premise application or service are available and working effectively for your users, instantly alerting you at the first sign of trouble.

Use Tevron’s CitraTest APM to:

  • Baseline application performance prior to cloud migration
  • Proactively monitor application performance post-cloud deployment
  • Be the first to know you have an application performance problem, slowdown or bottleneck
  • Resolve issues before large volumes of users are impacted
  • Validate your SLAs, from an end-user perspective, by application, by transaction step and by location
  • Keep everybody aligned and your business humming


Good luck with your cloud initiatives!