How to Develop Commercial-Grade Product

TO be continue

Indicators

Tech indicators

Security

Stability

Availability

  1. MTBF (Mean Time Between Failure)
  2. MTTR (mean time to restoration)
  3. Availability = (MTBF/(MTBF+MTTR))*100%

Performance

Operation indicators

Trading volume

CVR (Click Value Rate)

User account

PV/UV

Bounce Rate

Development Cycle

Requirements

BackLog

User Story

UISpec

Design

Design drawing

html & CSS

Develop

develop Environment:

  1. github Boxen for ios
  2. vagrantup

Version Manage Tools: git

Package Manager:

OS level

  1. BSD/Macos: homebrew(made of ruby& git)
  2. Linux
    • from source: configure/make/make install
    • using rpm/dpkg(hard mode): ==> centos/debian(ubuntu)
    • using yum/apt(easy mode): ==> centos/debian(ubuntu)

Language level

  1. .net: nuget
  2. java: maven
  3. sacla: sbt
  4. ruby: gem
  5. jsp: npm
  6. swift: spm
  7. python:
    • easy install (setuptools), can’t support uninstall
    • pip (good)
  8. go: get

Branchs Manage: master develop

Develop Process: Scrum

Engineering: CI (Continuous interaction) / CD(Continuous delivery) / platformization / Framization

Timer: 2 weeks

FrontEnd

BackEnd focus

High Scalability: timeout

Performance: cache/parallelization

Stress Testing tool:

  • TSung: TCPCopy
  • A/B test

debug

  1. keep context: no reboot, gcore/jmap/tcpdump
  2. restore context: tcpcopy,touchstone
  3. snap analysis: gdb、Xmap、mat、jstack
  4. call analysis: btrace、Xtrace
  5. composed ansysis: perf、Xstat、Xtop、sysdig

Security

Integration

tools

  1. jenkins
  2. Travis CI based on GitHub

process

one project, one build for small business

  • Project Architecture diagram
    image
  • develop process diagram
    image
  • Quality Assurance
    hand-test

    multi-projects, one build for mid business

    Module num > 20
    develop num > 100
  • Project Architecture diagram
    image
  • develop process diagram:
    commit role: from dev to bundle
    commit way: from committing code to repositories
    image
    image
  • Quality Assurance
    make code review platform
    make bundle platform
    make buider platform
    make test platform
    make insepection platform
    make gray release process
    image

    multi-projects, multi-build, pluginization for large business

    every Module is individual on the develp,code review,integration,test
  • Project Architecture diagram
    image
  • develop process diagram:
    image
    bundle dev phase
    integration phase
    release phase
  • Quality Assurance: CI,CD
    image

e.g

  • web develop process (from zhihu)
    1. Erect task on phabricator
    2. commit diff and relating to task
    3. unit test
    4. code reviewer
    5. merge code
    6. online test
    7. close task
  • android
    1. Genymotion (android emulator)
  • mobile quality assurance (from meituan)
    image
    image

Static resource optimization

Static code review

  1. regulation: google’s C++ coding style guide for all dev languages
  2. phabricator from facebook (good)
  3. gerrit

Develop output check

Build tools

  1. buildout for python
  2. gradle for android build
  3. maven

Version and config file

deploy tools

  1. fabric
    • made of python
    • <10 servers
    • servers status are same
  2. chef/puppet
    • made of ruby
    • slow
  3. salt/ansible (good)
  4. Docker (best,final)

route: BGP((Border Gateway Protocol), one IP, support accessing from Various Operators

CDN:

VPS(Virtual Privates Srever)

  • ailiyun
  • qingcloud
  • digitial ocean
  • linode
  • aws

Releases for various platform

Operation

improve abailability

  1. Avoid single point of failure:network/Device/code problem
  2. Narrow time of failure (imporant)
    • automate
    • one click
  3. Cure:
    • for stateless instance
      image
    • for state instance
      HA:master-salve
      Paxos/raft
      image
    • for cluster
      downgrade
      limit crowding
      automatic scaling
      fast fallback
    • for data center: multi-IDC
    • for local city
      image
    • for domestic city
      image
    • for global

Monitor and Warning

  1. zabbix+graphite(zhihu)
  2. xiaomi open-falcon
  3. supervisor

Log from frontend to backend

exception log
performance log
Visit log
Influence log
trace log
e.g
  1. zhihu’s log

Profiling everything

  1. for backend

    • gperftools
    • perf
    • gprof
      1
      2
      3
      gcc -pg a.c -o a
      ./a
      gprof ./a
  2. for frontend

User Actions

Data Analysis

Data Sample

PV/UV/BounceRate..

A/B Test

Recurrence

目录

  1. 1. Indicators
    1. 1.1. Tech indicators
      1. 1.1.1. Security
      2. 1.1.2. Stability
      3. 1.1.3. Availability
      4. 1.1.4. Performance
    2. 1.2. Operation indicators
      1. 1.2.1. Trading volume
      2. 1.2.2. CVR (Click Value Rate)
      3. 1.2.3. User account
      4. 1.2.4. PV/UV
      5. 1.2.5. Bounce Rate
  2. 2. Development Cycle
    1. 2.1. Requirements
      1. 2.1.1. BackLog
      2. 2.1.2. User Story
      3. 2.1.3. UISpec
    2. 2.2. Design
      1. 2.2.1. Design drawing
      2. 2.2.2. html & CSS
    3. 2.3. Develop
      1. 2.3.1. develop Environment:
      2. 2.3.2. Version Manage Tools: git
      3. 2.3.3. Package Manager:
        1. 2.3.3.1. OS level
        2. 2.3.3.2. Language level
      4. 2.3.4. Branchs Manage: master develop
      5. 2.3.5. Develop Process: Scrum
      6. 2.3.6. Engineering: CI (Continuous interaction) / CD(Continuous delivery) / platformization / Framization
      7. 2.3.7. Timer: 2 weeks
      8. 2.3.8. FrontEnd
      9. 2.3.9. BackEnd focus
        1. 2.3.9.1. High Scalability: timeout
        2. 2.3.9.2. Performance: cache/parallelization
        3. 2.3.9.3. debug
        4. 2.3.9.4. Security
    4. 2.4. Integration
      1. 2.4.1. tools
      2. 2.4.2. process
        1. 2.4.2.1. one project, one build for small business
        2. 2.4.2.2. multi-projects, one build for mid business
        3. 2.4.2.3. multi-projects, multi-build, pluginization for large business
        4. 2.4.2.4. e.g
      3. 2.4.3. Static resource optimization
      4. 2.4.4. Static code review
      5. 2.4.5. Develop output check
      6. 2.4.6. Build tools
      7. 2.4.7. Version and config file
      8. 2.4.8. deploy tools
      9. 2.4.9. route: BGP((Border Gateway Protocol), one IP, support accessing from Various Operators
      10. 2.4.10. CDN:
      11. 2.4.11. VPS(Virtual Privates Srever)
      12. 2.4.12. Releases for various platform
    5. 2.5. Operation
      1. 2.5.1. improve abailability
      2. 2.5.2. Monitor and Warning
        1. 2.5.2.1. Log from frontend to backend
          1. 2.5.2.1.1. exception log
          2. 2.5.2.1.2. performance log
          3. 2.5.2.1.3. Visit log
          4. 2.5.2.1.4. Influence log
          5. 2.5.2.1.5. trace log
          6. 2.5.2.1.6. e.g
        2. 2.5.2.2. Profiling everything
        3. 2.5.2.3. User Actions
    6. 2.6. Data Analysis
      1. 2.6.1. Data Sample
      2. 2.6.2. PV/UV/BounceRate..
      3. 2.6.3. A/B Test
    7. 2.7. Recurrence