Tackling the IP Address Crisis: Alibaba’s Youku Leads IPv6 Adoption

As the world faces a shortage of IPv4 addresses, Youku is pioneering the transition to IPv6

Youku’s Rollout

Youku’s plans for IPv6 started early in 2018. During the World Cup, Youku investigated the technical feasibility of IPv6 and prepared and verified its transition plan. Then, during the 11.11 Global Shopping Festival, IPv6 was rolled out to a selection of Youku PC and app users in the Chinese domestic education network and in key front-line cities. With the IPv6 technology, these users were able to enjoy live, on-demand HD services. This user experience could be likened to driving on a newly paved highway without other traffic to slow down for. Users had access to exclusive user identities, exclusive client network detection functions, exclusive membership cards, and exclusive “red packet” bonuses.

Why upgrade to IPv6?

IPv6 offers a number of advantages over IPv4 for Youku. Crucially, IPv6 has a larger-capacity address space that does not have the same address shortage as IPv4 and allows true stateless address autoconfiguration. This overcomes the current lack of IPv4 address availability and enables unlimited expansion of services. In addition, the header format used for IPv6 is greatly simplified, reducing the processing overhead for the header. This capacity increase and reduced overhead are crucial for the effective deployment of an “Internet of Everything” approach to the Internet of Things.

The transition plan

Youku’s IPv6 transition plan spans three phases:

Scope of the IPv6 transition

The overall scope of IPv6 transition is large, with the following factors comprising the entire link:

IPv6 Transformation for Developers

For Youku developers, transition of the client app/PC webpage and the server were the sole areas of concern.

Client

IPv6 requires the update of second-party libraries such as the basic network package NetworkSDK. For clients that use httpdns to parse, they needed to be updated so that the client network determined and received the AAAA record sent by the httpdns server. For clients that use localdns to parse, they needed to be updated so that the identification of the AAAA record resolution was added to the DNS service request parameter. For clients that use third-party libraries, IPv6 must be supported, which involves updating the libraries to newer versions that support IPv6 where possible. However, in some cases this was not possible and developers replaced the libraries instead.

PC/server

The PC/server area involved more potential issues and need for transition work.

Challenges Encountered During the Transition

As a pioneer in IPv6 transition, Youku encountered a number of problems during the transition, and was prepared to face various unexpected challenges while innovating in this space.

Lack of available IPv6 environments

Initially, the office network did not have an IPv6 access environment, hindering service development.
The intranet had not yet been transitioned, limiting access to the daily (test) environment. At the beginning, when the basic environment was not available, Youku used an IPv6 over IPv4 link VPN to connect to the test environment. To implement that, certificates needed to be added to the PC/client and hosts needed to be changed. However, hosts could not be changed on mobile terminals without getting administrator privileges illegally. This initially caused issues, but it was still possible to start the service tests.

OS network modules

Since containers must support obtaining IPv6 addresses from request headers, the user IP must be passed through level by level. As a result, the network modules at various levels of the server needed to be upgraded and the message header needed to be expanded as well. For example, the toa module that is used to allow the backend realserver to see the real clientip rather than the lvs dip had to be upgraded.

Special address library requirements

Address libraries are not uniform across Youku. Youku has its own regional code; the Youku advertising service uses regional code provided by the China Advertising Association, and other services use Alibaba Group’s address library.

The MTU problem

In the IPv4 era, devices on the three layers of the intermediate network would be sharded. As a result, the MTU was generally set to a maximum value of 1500 to reduce network overhead. However, to reduce complexity and the costs of devices for intermediate network layers, these devices are no longer sharded in the IPv6 protocol and are specified during negotiation between the two ends.

Verifying whether a client is using IPv6

This challenge presented a serious concern. When a network has begun using IPv6 and the service is running normally, it must be confirmed that the network is running on IPv6 and has not been downgraded. Capturing client logs is one approach to this, and the most accurate. However, Youku was already working to transform the client, so instead integrated a way to verify the network’s use of IPv6 directly into the client.

Protocol rollback

  • In a dual-stack environment, the node can theoretically communicate with any optional protocol.
  • Most operating system protocol stacks and application implementations will default to IPv6.
  • When the IPv6 path is “unavailable”, it can automatically roll back to IPv4.
  • But the older rollback algorithm affects user experience to a large extent.
  • Adopt a “more aggressive” rollback algorithm
  • For the mobile client, Youku have greater control
  • For the PC client, the situation is complicated and needs to be treated carefully and fully tested.

Security issues

The carrier’s export function is currently available. However, black hole capability is still in development and currently unavailable.

CDN canary release

The CDN domain name is controlled by Alibaba Cloud and cannot be in the same scope as the canary release of the service. Youku added IPv6-exclusive CDN domain names to sidestep this issue. By adding business logic on the service side, they could issue different domain names to match the pace of the canary release.

Confirming the Service is Running Normally

There are a number of different parts of the platform and service that Youku had to confirm were running normally during testing and canary release — namely the service layer, access layer, data platform, and basic network.

Looking to the Future

As Youku continues to roll out its IPv6 transition, its development team is looking to what it allows them to do in the future. With IPv6, Youku removes the bottleneck problem of IPv4 so that the number of devices people use is no longer an issue. Every device can be interconnected to form a networked Internet of Everything, making it easier than ever before for Youku to work with partners across the industry and improve the experience for end-users and content owners.

Alibaba Tech

First hand and in-depth information about Alibaba’s latest technology → Facebook: “Alibaba Tech”. Twitter: “AlibabaTech”.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Tech

First-hand & in-depth information about Alibaba's tech innovation in Artificial Intelligence, Big Data & Computer Engineering. Follow us on Facebook!