Prospective RC Broadband Project Technical Summary
Basic Technical Summary
Here is an overview of how the technology works.
Fiber Optic Cable
The glass strands are about the thickness of a hair, with a tiny “core” of pure glass surrounded by an outer layer of glass, and surface glaze. There are two types, single-mode and multi-mode. Single-mode has a small core for a laser aimed straight through, there are ~30 types, mostly long range (miles). Multi-mode has a wider core for multiple lasers bounced off the outer glass layer walls, there are a ~dozen types, mostly short range (LAN’s).
Strands are encased in plastic, used as-is (patch cables) or grouped into bundles. Bundles are encased in plastic, usually with a reinforcement, or with other bundles in a larger cable. Direct burial cable is armored with corrugated metal. Aerial cable has a reinforcement cable (steel or kevlar) in either the cable center or above in a “figure 8”.
Cable “core” (strand) counts scale with cost, for a county-wide network 48-core seems to be the upper-end of economical investment in future capacity. For special lines 288-core is available in similar aerial or buried configs, and folded ribbons of up to 1000-cores wrapped in a soft loose cable jacket.
For buried cable, HB3093 requires use of conduit, to be shared with other projects, with this the cable costs a little less but conduit is much more. Cables should be buried whenever possible, especially backbones, but the labor and trench equipment costs are much higher, and installation is much slower ~ e.g. double budget, double time.
Active Optical Network (AON)
This is a point to point segmented network. Single-mode fiber connects one piece of equipment directly to another piece of equipment, like switches, routers, PON network boxes, computers, line repeaters, etc. Line length is tens of miles ~ varies by equipment used and fiber type & quality.
Equipment cost scales with speed, up to 1Gbps is very cheap, up to 10Gbps is cheap for a low-end backbone, up to 100Gbps is relatively easy yet increasingly expensive off-the-shelf hardware, that’s probably our upper limit for now. Speeds of 100Gbps to ~400Gbps requires very high quality cable with special hardware, ~400Gbps to ~1.5Tbps is mainly used for intercontinental backbones (mostly undersea) with special hardware, and above ~1.5Tbps only exists in a labs. There’s a point were adding cores to cable costs less than more speed per core.
Networks: For this project, three county-wide fiber networks are needed ~ “external”, “internal”, and “government”, but they can share cable where routes overlap.
- County External Network: Connects to neighboring county networks and internet backbones. Most would run straight from the county-line to Spencer. This should be buried, if financially possible.
- County Internal Network: Connects to all ISP customers, either directly or PON box.
- County Government Network: Connects all county buildings, and public safety related stuff. With dedicated fiber this can be physically isolated from the other networks. Should be buried.
- Dark Fiber: Build in extra capacity to lease for extra income. Note, buried fiber is worth more.
Penta-Route: These five lines totaling ~93 miles are the major backbones serving the county, from a hub somewhere in or near Spencer to some point near each of...
- 36S to Wallback, ~30 miles, link to Clay Co network, maybe Frontier fiber.
- 33E to Arnoldsburg, ~14 miles, link to Calhoun Co network.
- 119S to Cotton, ~24 miles, link to Kanawha Co network, maybe WV Fiber Net.
- 33W to county line, ~13 miles, link to Jackson Co network.
- 14N to county line, ~12 miles, link to Wirt Co network, maybe Lumos fiber.
Branch-line Backbone Bridges: These are chains of branch line segments with one or more fibers dedicated to linking far ends of one penta-route line to another for internal network redundancy. A later upgrade with more expensive routers could add external network redundancy as well. Another possibility is secondary internet gateways to decentralize from Spencer, but that would require a lot of expensive network equipment.
Passive Optical Network (PON)
This is the primary method used for deploying single-mode Fiber To The Home/Premises/Building/Farm (FTTx), as follows… At a point along a backbone a network box (powered) has multiple PON ports, a PON line is run from that box along a route up to about 10 miles long, with up to ~10Gbps bandwidth (per port). Along the PON route optical splitters are used to attach drop-lines to each house. The optical signal goes out to all houses on the same PON port, multiplex channels and encryption keep traffic for each house private, and all houses share the same bandwidth per port. Split ratio is the number of houses sharing the PON port, each house added reduces signal strength, max line length, and available bandwidth, but shares the network cost among more users, increasing profit. That those splitters are cheap and not powered makes PON extremely cost effective versus AON for wiring neighborhoods or buildings (another usage is Fiber To The Desk or Computer).
Types of PON & Notes: There are about a dozen types of PON, fiber is the same, only the equipment attached to it varies, four are in common usage in the US. GPON vs EPON is an ongoing industry tech struggle.
- Broadband PON (BPON): Replaced APON, is used by legacy telco data+voice service, its obsolete but may exist in the county as part of Frontier’s DSL backhaul, upgrades to GPON.
- Gigabit PON (GPON): Used by ISP’s with triple-play service (e.g. FiOS), mostly in the US, up to 2.5Gbps at the PON port, up to 1Gbps service, upgrades to 10GPON (10Gbps). The underlying network supports ATM & frame-relay, such as T1 DSU phone lines (legacy telco baggage). GPON is better than EPON for standard 1Gbps service due to the faster GPON port speed (up to 2.5Gbps), better phone service (via ATM not VoIP), better for non-IP video (e.g. FiOS TV), and better network management features.
- Ethernet over PON (EPON aka 1G-EPON): Used in most of the rest of the world, up to 1Gbps, is now cost competitive with DSL (copper) at the same bit rates (e.g. 25Mbps) in very high split-ratio networks (e.g. 128 houses per port, 128:1), upgrades to 10G-EPON. Usually deployed as Carrier Ethernet (CE2.0, CE1.0, previously MetroEthernet), or a similar expanded service platform, but is fundamentally the same as home/office LAN ethernet. EPON is better than GPON for roughly half equipment cost at 1Gbps service, up to 90% lower equipment cost at 25Mbps service, uses less than half the energy (good for battery backup), is technically less complicated (GPON is multiple networks in one platform + legacy baggage), is more bandwidth efficient for IP streaming video (i.e. all online video, e.g. Netflix Hulu, YouTube, etc).
- Radio Frequency over PON (RF-PON): Cable TV industry adaptation from coax cable to PON, its multiple networks in one (like GPON), provides best TV support. Probably used by SuddenLink.
- 10GPON & 10G-EPON: New, expensive, so far it looks to continue the pattern, 10G-EPON looks to be less expensive while 10GPON is probably better for UltraHD TV yet loses its port speed advantage. For us 10Gbps PON service is at least a decade off (10Gbps leased AON lines is different).
PON Equipment Platforms: There are 45+ platforms from 20+ manufacturers, but new ones are created, old ones obsoleted, and the companies buy each other at such rates its difficult to keep track. Most have proprietary variations of a standard PON type. Most also provide an integrated software platform, for network management, services management, accounting, customer service, etc.
Utility Pole Access
Utility pole access is a major cost in running aerial cable, both in “make-ready” cost, ongoing rent, and inevitable repair costs from storm damage, but there is insufficient information to include this in the Project Costs section.
Make-Ready Cost: A section of the National Broadband Plan which cites FiberNet average costs of $4,200 per mile and estimate of a cost reduction to $1,000 per mile with simplified make-ready procedures. If that’s accurate for us, it would add roughly $93k to the Penta-Route, and several times that to the branch-lines and PON lines. However, the report doesn’t say if that’s rural, urban, major road, second or tertiary roads, etc.
Pole Rent: The National Broadband Plan noted typical rent rates of $7 for cable operators and $10 for telephone lines, per pole per month. Its hoped this can be offset, or significantly reduced, by right-of-way maintenance.
Pole Ownership: Another site noted in some areas power companies and telephone companies try to own poles in equal numbers so the net rent is neutral, and in some cases pole owners are free to deny others access. If that’s true here, it may be difficult to get access to Frontier’s poles.
Combined with ongoing rent, the notion of buried conduit is a little more appealing, and might be cost effective.
Micro-Trench Buried Conduit
HB3093 defines microtrenching as using a cutting wheel type of trencher to cut a trench up to 3 inches wide and 1 to 2 feet deep. It provides reduced permitting requirements, in exchange it requires installation of at least double conduit capacity for use by third-parties, and precise maps reported to BEC.
Micro-Trench Equipment: Several brands are available, including Ditch Witch MT16 and Marais RT100, but some small rock-wheel trenchers (aka rock saws) may work too (sort mounted on Bobcats or backhoe arms). Useful pricing estimates are difficult to pin down, guessing in the $30k to $50k range, including tractor. Similar size chain type trenchers are a third less, use these in fields/yards where there is no benefit to the wheel type.
Conduit: High-density polyethylene (HDPE) pipe, nearly the same as orange gas pipe. Can be armored to protect against rodents. §31G-3-2-c says vacant conduit of the same size must be installed, so maybe a 1.5 inch trench with two ~3/4 inch (OD, ~1/2in ID) HDPE pipes. Pricing needs further research, estimate $0.25 per meter x 2.
Right-Of-Way (ROW): Most of this will run along state and county roads, and member property, its assumed ROW will be granted freely. Some funds should be budgeted for ROW related costs, needs further research.
Conduit Map Problem: GPS fix is insufficiently accurate. Could use a survey team with laser range finding equipment to manually find fixes to various permanent landmarks. Another possibility is open source robotics 3D video spacial positioning software using a Xbox Kinect, or similar arrangement of cameras and laser range finder.
Generally speaking, the more services provided from within the county, the more money will be kept local, more local jobs created, and the more benefit it will have to the cooperative membership.
Email Services: Good email is a critical, most ISP’s have lousy email.
Telephone Services: There are two basic types of service, Public Switched Telephone Network (PSTN), and Private Branch eXchange (PBX), PSTN has a high cost with heavy regulation but is what most people will expect. The hardware included with most PON platforms can provide analog Plain Old Telephone Service (POTS) as PSTN and PBX, as well as Voice over IP (VoIP). Lots of features can be added with server software, including all common PSTN and PBX features, use of VoIP to extend phone service to nearly any network attached device with audio input & output, and communications integration, like fax to email, print to fax, voice mail to email, text messaging, apps/software for various devices, etc.
- Co-op PBX Phone Service: Include with basic FTTx PON service, so all members are assigned an extension number. Offer a “dedicated line” add-on to forward a PSTN number to their extension.
- Virtual PBX Phone Service: Two offerings, a subset of the co-op PBX with local extensions, and larger PBX server (VPS). Replaces older office phone systems, is cost accessible to most households.
- PSTN Phone Service: Provide regular phone service to members. This is a costly upgrade, requires the co-op to become a Competitive Local Exchange Carrier (CLEC), and comply with a complicated mess of FCC and state regulations. Needs more research.
Community Information & Support Service: The co-op may find it worthwhile to fund a service for members to have someone to call, text, or email with nearly any question they need answered, or to report anything of community interest. Think of it as a variation on 311 municipal information services, a sort of county nervous system, community knowledge infrastructure, and source of influence for the co-op.
Security Systems, Life Line, Monitoring: Off-the-shelf hardware, installation, maintenance, and monitoring services. Its relatively easy service to offer locally, can integrate with other communications services, and be better connected with local emergency services for faster response than outside companies.
Mix of direct or reseller offered services sufficient to replace cable/satellite TV service for most people. Some people will want more, lots of fairly inexpensive IPTV services are available online.
Streaming Video: Special servers which offer Video On Demand (VOD) from files, relayed remote network feeds, and relayed satellite feeds, all with multiple interfaces to find & watch the desired video. Should include co-located servers from other content providers, such as Netflix’s Open Connect, which allows ISP’s to cache popular video on a server within their network to save backbone bandwidth usage, as well as to sell the service to their customers.
IPTV: Streaming video turned into a scheduled channel service vaguely similar to cable TV, including locally popular streams from co-located online sources (Netflix, Hulu, etc), but can include community and member streams. Use of IPTV has two purposes, first is to provide a traditional channel-schedule TV experience including set-top boxes like DVR’s (or HTPC with MythTV); and second to increase network bandwidth efficiency via consolidating lots of individual VOD streams into a single multi-cast stream. This does not fit the definition of cable TV, so it should not be subject to that regulation.
Cable TV: Service like FiOS, equivalent to cable TV or satellite TV. Requires GPON network. Attaches cable TV rules and regulations of WV code §24D and FCC.
Web & Security Cameras: Video feeds accessible from anywhere, including as a IPTV channel, stored in the data center, and shared publicly on a county portal, or secured with access controls & encryption. Variations like traffic cam with a vehicle counter, weather cam with sensors, or river cam with height reading can include extra data feeds collected to a database. Object recognition can detect people, vehicles, or critters (e.g. deer), in places they shouldn’t be and trigger an alarm or other instruction.
Web Hosting Services
There are several niche usages where local web hosting is advantageous and profitable. Other services can be offered in partnership with third-parties when its cost ineffective to do so locally.
Community Social Media Platform: Free open source social platforms are available, these can be tightly integrated into other communications systems, and linked to global open social networks.
Cloud Storage Service: Provides network storage, gallery, address book synchronization, etc, including file sharing. Intended for personal, family, or small office. For example, a Nextcloud server.
Shared Web Hosting Services: A variety of service packages on a hybrid platform, part local servers, part white-label reseller hosting from large web hosts, and a management platform which handles both as a pair of server clusters (either cPanel + WHM or Virtualmin + Webmin). Locally focused sites would be hosted on local servers, sites with little local traffic on third-party servers, large sites mirrored on both.
Virtual Private Server (VPS) Hosting: Linux VPS servers, general purpose, such as a custom web server, various network appliances, multiplayer game servers, and local business’s back-office servers. This would allow local businesses to externalize much of their server hardware, and get better results at a lower cost. Can be used as a remote desktop too, accessible from low-end PC’s or low-cost thin clients (e.g. tablets, “TV stick”, Chromebooks, etc), but are limited to Linux and no 3D games.
Business Systems Hosting: There are a variety of free open source website packages and server software which can easily & inexpensively add IT capabilities to companies, improve efficiency, free up time, and save money. Including groupware, customer relationship management, accounting, enterprise resource planning/management, knowledge base, help desk, and more. The most common problem with these is a lack of technical experience for the business to use them effectively, and lack of the hosting provider to provide the needed support, this is a niche where we can recruit local people to provide that support to local businesses (instead of someone in India).
Backup Service: Cheapest service would be with a third-party storage partner. Co-op backup would be faster. The slightly paranoid could opt for both. We could add a hardened storage bunker somewhere in the county, basically a buried water-tight concrete box with shock-mounted servers, SAN, backup power, and redundant FTTx lines.
Dedicated Servers & Co-location: Early on only offer in special cases, later add a second data center for other people’s equipment. Co-location is leasing cabinet space in a data center for others to provide their own equipment, and server sales. Dedicated servers is equipment rental. Both include physical tech support, but server administration costs extra. Both create local jobs for server techs and server admins.
Content Delivery Network (CDN): Offer local node hosting to CDN operators, this will make popular content available on the co-op network with high performance and reduce backbone traffic. Ordinarily CDN’s pay for hosting on larger networks, but with this small a user base the co-op may need to pay them.
Network Caching & Mirrors: Special servers with an enormous amount of high speed memory and storage (PCI-E RAM disks and RAID SSD) can cache popular content to improve network performance. File mirrors are similar. Notable for being another type of network infrastructure, can be expensive but will pay for itself in reduced backbone load, improved performance, and happier users.
Network Intelligence: A few servers scan everything attached to the network, develop profiles of those devices, discover services offered, index any content offered, and maintains a county network search engine. Profile data is analyzed for technical problems, security risks, viruses & malware, traffic management, overall network usage metrics, etc. Members can assist by running monitoring software which reports network metrics, and detects probing by intruders (hackers). To offset this inherent capacity to spy on users, educate members on safeguarding their own privacy and internet security. Additional privacy related services could be offered. Traffic content will not be monitored.
Super Computer: Local distributed progressing super cluster service, similar to Folding@Home (cure cancer) and SETI@Home (aliens), but for any local needs, or member’s choice of good cause.