Migrating Your Digital Fiber Management System of Record Requires Data Science and Translation Teamwork
The world of telecommunications is in the midst of historic growth. It’s hard to avoid the ongoing wealth of news about the activity around planning, building, and operating the next generation of fiber optic infrastructure. It’s thrilling to be in the business of fiber with such strong tailwinds supporting a more connected future.
One long-term trend that has enabled the recent exponential interest in building fiber is the advancement of fiber technologies that make it easier to deploy accessible and profitable high-speed fiber-to-the-home (FTTH) networks to underserved areas.
Along with the equipment, cables and construction techniques that go into these next-generation networks, the software systems that enable deployment have also gone through exciting transformations. Digital fiber management systems as a software category represent a broader, more robust suite of tools than they did even just a few years ago. The continuous delivery of new features, upgrades and performance enhancements have become the standard of true SaaS fiber management. The result is a high-energy environment of constant, user-focused evolution.
Communications Service Providers (CSPs) of all shapes and sizes are adopting GIS fiber management systems (FMS) to maximize profits, enhance productivity and improve control and visibility of their network. In most cases, these operators have a need to migrate legacy network data from multiple sources, formats and use cases into the established format, or “schema,” of their chosen FMS.
Some network data can be migrated easily, or even imported by the user in some platforms—like .kmz or .shp files. However, data stored in .csv, .pdf, CAD or other less GIS-ready formats will require some work to prepare for migration into the more precise world of digital fiber management. What follows are five recommendations for approaching this work, made during an ISE Webinar, “How to Prepare your Legacy Data for Digital Transformation.” (You can watch the full webinar on demand here: https://www.isemag.com/home/webinar/14282147/november-3rd-webinar-speed-accessibility-and-productivity-how-to-prepare-your-legacy-fiber-management-data-for-digital-transformation.)
1. Start Earlier Than You Think You Need To
Don’t let the ghosts of unwieldy, past-deadline and over-budget data migration projects haunt you. Keep in mind that along with all the other cool tech improvements going on, the process of digitizing has evolved too. Every fiber management software vendor will expect some level of data work to get you started in their system and will have their own stack of tools and best practices for bringing your network in properly—all you need to do is show up prepared.
Starting early to clear the road for the eventual move can accelerate the extract-transform-load (ETL) process and get you productive in your new FMS quickly. If you’re thinking about implementing an FMS within the next six months, now is the time to dig into your data and get prepared. (See Figure 1.)
2. Know Where You Are… And Where You Want to Be
The process of ETL is a lot like translating a text from one language to another. It’s rarely perfect on the first try, and requires not just editing, but the development of rules that give direction on how certain sets of data should be used.
That’s where the human element of digitizing plays a big role. Your migration project will benefit from project managers that know your network in and out. An intimate knowledge of the “source text” and a vision for how the “translated” version will be handled in your FMS are the two key elements to a smoother transition. Your project leader should have a solid understanding of both.
Along with the clear sense of where you’re starting and where you want to be, your migration process should consider three core basics of good data science:
3. Start Small, Then Repeat What Works
Applying data science to information collected and documented through real-world, human-led processes is inherently iterative—people make mistakes and deviate from process, while software reins people in by requiring accuracy and completion.
Your FMS data migration team will be focused on gathering information about how your processes inform your data. They will then use that information to customize the tools that will translate your data to the schema of the FMS into which you’re moving; but to get there, it helps to start with a small data set.
Breaking your network data down into smaller chunks gives you the opportunity to create, test, inspect and refine your documentation and migration rules efficiently, then use the results of that process as the foundation for migrating the remaining data. We refer to this stage of migration as the rapid prototyping stage.
Getting that one accurate area established up front is often a good strategy. Once patterns and consistency of source are pinned down enough to execute on a single area, good data science partners are going to be fanatically focused on repeatability and automations—so once you break through that first neighborhood, the rest come fast.
How you break up your network data is up to you but focusing on prioritized areas that you want migrated first, or that serve as the “ideal” example of your data as a whole, can be helpful in determining your approach.
4. Get Your Names Straight
In the world of fiber, there can be a lot of different names for the same thing. What one FMS classifies as an “enclosure” might be called a “pedestal” in your own records, for example. These differences between your existing schema and that of your FMS are the details that you will map out early on in your work with your data migration team as they familiarize themselves with your documentation.
This translation work is a major part of the initial process to map out how your data should be represented in your FMS. From there, the migration process will demand nothing so much as consistency (stick to the rules you just created). Ensuring that each instance of the same thing in your documentation is referred to in the same way, with the same details, will increase the speed of the translating process by removing ambiguities that trip up ETL tools.
Review all the network data you plan to migrate and triple-check for little things that you might not normally notice. For instance, if one splice table indicates the presence of a splitter with “SP” while another uses “splitter,” your team’s role is to choose one standard term and propagate it throughout all your documentation. This will ensure every splice table has the same format and follows the same naming conventions and level of detail.
5. Be Ready to Explain Your Data’s Relationships
The last crucial part of prepping your data for migration into an FMS is identifying how you tie your connectivity data to your network geometry and ensuring that these relationships are documented consistently and completely throughout your network data. In our onboarding process, we refer to these elements that dictate how your network data ties together as linked object identifiers.
Your vendor’s ETL tools will use these identifiers to ensure that, as an example, the right splitter ends up in the right-hand hole in the right neighborhood. Often, operators will already have a lot of this relational information built into their data and the process is one of clarifying and creating consistency in collaboration rather than creating new rules. Linked object identifiers can be anything from a simple append to text data, the presence of ID numbers, or really any other in-house method to which your organization adheres. This aspect of your network migration, in particular, will benefit from close involvement from your organization’s legacy data owners to explain how the relationships between network objects should be interpreted in a visual model of your network.
“Don’t let the ghosts of unwieldy, past-deadline and over-budget data migration projects haunt you. Moving data into a new system is a bit like moving into a new house—consider it an opportunity to leave all the old stuff you didn’t really need behind and arrange things in a new and streamlined way. It is always going to be work, but identifying what is really important lets you shift into the new operating paradigm of the new system, which you selected for good reason. Getting the ‘move’ over is worth it to get into that new house!” Will Mitchell, Co-Founder and CEO, VETRO, Inc.
Take it Step by Step
Remember, you don’t have to show up to your first migration meeting with everything perfected and ready to ingest all at once. It’s common to use a “sequential ingest scenario” in which operators choose an active area with data they feel confident in to start their project. This allows the user to get value from the new FMS from day one, while reserving older, less critical data for the back half of your migration.
Taking steps to move from legacy systems to a digital system of record may seem like a daunting project. Assigning a project manager, identifying how you want your network to be visualized and represented in your FMS, and working with an experienced partner can make the transition easier. As you prepare to digitize your network, remember that you can start small and still make big gains. If your old legacy system and messy documentation is slowing your business down, bite that bullet and get going in a modern, integrated FMS. The old data can be worked in even while you are designing and digitizing a new network in parallel and enjoying the efficiency and collaboration benefits of the new system of record. That shift to a better platform and way of working is well worth the effort!