序言：我们上传大文件的时候，往往会上传失败。对多数情况下，修改配置文件即可使用。可是这样往往不能很好的解决对于大型文件的上传。比如1GB的视频文件。这时候就需要我们将文件切分成一个个小文件来上传。最后在进行重新的整合。
以thinkphp5.1为例：
我们设定一个场景：我需要上传一个大于1G的音频、或者视频文件，并且要上传到第三方服务器。那么、第一步：我们需要将资源文件切片先上传到自己的服务器上，然后重新整合为一个文件。第二部：将其上传到第三方（也是断续上传）。
代码示例：
1.前端使用 webuploader
html 代码大致为：
<link rel="stylesheet" type="text/css" href="/css/plugins/webuploader/webuploader.css">
<script src="/js/plugins/webuploader/webuploader.min.js?v=2"></script>
<div class="form-group">
<label class="col-sm-4 control-label">视频上传：</label>
<div class="col-sm-8">
<div class="wu-example">
<!-- 用来存放文件信息 -->
<div id="video_content" class="uploader-list "></div>
<div class="btns">
<div id="picker_video">选择视频</div>
<div id="upload_video" class="btn btn-primary hidden">开始上传</div>
<div id="cancelFile_video" class="btn btn-default hidden">取消上传</div>
<input type="hidden" name="videoPath" id="videoPath" value="">
</div>
</div>
</div>
</div>
页面js为：
fileupload({
server: "{:url('/course/upload')}",
progressId: 'video_content',
type: 'video',
pickerId: 'picker_video',
uploadId: 'upload_video',
cancelId: 'cancelFile_video',
chunked: true,
path: $('input[id="courseId"]').val(),
success: function(data) {
$("#videoPath").val(data.path);
}
});
fileupload方法所用的js脚本为：
window.fileupload = function(options) {
var config = {
server: '', // 上传服务器地址 必填
cancelServer: '', // 取消的服务器地址
progressId: '', // 进度条容器 必填
type: '', // video audio 默认为图片
pickerId: '', // 选择文件ID 必填
title: '文件上传', //
auto: false, // 自动上传
multiple: false, // 多选
chunked: false, // 分片
uploadId: '',//上传id
cancelId: '',//取消id 视频音频有，图片没有
path:'', // 子目录
imgType:'', // 上传图片类型
data: {
}, //取消上传的参数
success: function(data) {
console.log(data);
}
};
$.extend(config, options);
var _extensions = 'gif,jpg,jpeg,bmp,png';
var _mimeTypes = 'image/*';
if (config.type && config.type == 'video') {
_extensions = '3gp,mp4,rmvb,mov,avi,mkv';
_mimeTypes = 'video/*';
} else if (config.type && config.type == 'audio') {
_extensions = 'WMA,MP3,MIDI';
_mimeTypes = 'audio/*';
}
var $list = $("#" + config.progressId);
var _file = '';
var uploader = 'uploader' + config.progressId;
uploader = WebUploader.create({
auto: config.auto, // 是否自动上传
pick: {
id: '#' + config.pickerId,
name: "file", // 这个地方 name
// 没什么用，虽然打开调试器，input的名字确实改过来了。但是提交到后台取不到文件。如果想自定义file的name属性，还是要和fileVal
// 配合使用。
// label: '选择文件',
multiple: config.multiple
// 默认为true，就是可以多选
},
swf: '/js/plugins/webuploader/Uploader.swf',
// fileVal:'multiFile', //自定义file的name属性，我用的版本是0.1.5 ,打开客户端调试器发现生成的input
// 的name 没改过来。
// 名字还是默认的file,但不是没用哦。虽然客户端名字没改变，但是提交到到后台，是要用multiFile 这个对象来取文件的，用file
// 是取不到文件的
server: config.server,
duplicate: true, // 是否可重复选择同一文件
resize: false,
formData: {
"status": "file",
"contentsDto.contentsId": "0000004730",
"uploadNum": "0000004730",
"existFlg": 'false',
"path":config.path,
"imgType":config.imgType
},
compress: null,
chunked: config.chunked, // 分片处理
chunkSize: 50 * 1024 * 1024, // 每片50M,经过测试，发现上传1G左右的视频大概每片50M速度比较快的，太大或者太小都对上传效率有影响
chunkRetry: 2, // 如果失败，则不重试 2为失败可以重复
threads: 1, // 上传并发数。允许同时最大上传进程数。
// runtimeOrder: 'flash',
disableGlobalDnd: true,
timeout: 600000,
accept: {
title: config.title, //文字描述
extensions: _extensions, //允许的文件后缀，不带点，多个用逗号分割。,jpg,png,
mimeTypes: _mimeTypes, //多个用逗号分割。,
}
});
// 当有文件被添加进队列的时候
uploader.on('fileQueued', function(file) {
_file = file;
$list.html('<div id="' + file.id + '" class="item">' +
'<h4 class="info">' + file.name + '</h4>' +
'<p class="state">等待上传...</p>' +
'</div>');
if(config.cancelId){
$('#'+config.uploadId).removeClass('hidden').siblings('#'+config.cancelId).addClass('hidden');
}else{
$('#'+config.uploadId).removeClass('hidden');
}
});
// 文件上传过程中创建进度条实时显示。
uploader.on('uploadProgress', function(file, percentage) {
var $li = $('#' + file.id),
$percent = $li.find('.progress .progress-bar');
// 避免重复创建
if (!$percent.length) {
$percent = $('<div class="progress progress-striped active">' +
'<div class="progress-bar" role="progressbar" style="width: 0%">' +
'</div>' +
'</div>').appendTo($li).find('.progress-bar');
}
$li.find('p.state').text('上传中');
$percent.css('width', percentage * 100 + '%');
});
uploader.on('uploadSuccess', function(file, data) {
if (data.error) {
alert(data.error.message);
$('#' + file.id).find('p.state').text('上传出错');
config.cancelId ? $("#"+config.cancelId).addClass('hidden') : '';
return;
}
$('#' + file.id).find('p.state').text('已上传');
if (config.success) {
config.success(data);
}
if(config.cancelId){
$("#"+config.cancelId).addClass('hidden');
}
});
uploader.on('uploadError', function(file) {
$('#' + file.id).find('p.state').text('上传出错');
if(config.cancelId){
$("#"+config.cancelId).addClass('hidden');
}
});
/** * 验证文件格式以及文件大小 */
uploader.on("error", function(type, handler) {
if (type == "Q_TYPE_DENIED"&&config.type=='video') {
alert('请上传正确格式的视频!(如mp4)');
}else if(type == "Q_TYPE_DENIED"&&config.type=='audio'){
alert('请上传正确格式的视频!(如mp3)');
}else{
alert('请上传正确格式的图片!');
}
if(config.cancelId){
$("#"+config.cancelId).addClass('hidden');
}
});
uploader.on('uploadComplete', function(file) {
$('#' + file.id).find('.progress').fadeOut();
});
if(config.cancelId){
$("#"+config.cancelId).on('click', function() {
uploader.cancelFile(_file);
$('#' + _file.id).find('p.state').text('上传出错').end()
.find('.progress').remove();
$(this).addClass('hidden');
$('#'+config.uploadId).addClass('hidden');
$ajax(config.cancelServer, 'get', config.data, function(data) {}, function() {});//取消执行的请求
});
}
$("#"+config.uploadId).on("click", function() {
uploader.upload();
$(this).addClass('hidden');
$('#'+config.cancelId).removeClass('hidden');
})
}
2.后端控制器的代码：
我从前端传入了用于制作子目录的参数，如我将课程的id作为子目录。并且通过判断文件的类型，将视频、音频文件分开存放。
/**
* 大文件分段上传
* @return string
*/
public function uploadFile(){
$path= input('path'); // 课程id用于制作子目录
// 判断文件类型
if (isset($_REQUEST["name"])) {
$fileName = $_REQUEST["name"];
} elseif (!empty($_FILES)) {
$fileName = $_FILES["file"]["name"];
} else {
$fileName = uniqid("file_");
}
$exeArr = explode('.', $fileName);
if(is_array($exeArr)){
$exe = end($exeArr); // 文件后缀
}else{
die('{"error" : {"message": "文件类型错误"}}');
}
$UploadService = new UploadService();
$videoExe = $UploadService->videoExe; // 视频类型数组
$audioExe = $UploadService->audioExe; // 音频类型数组
// 判断服务器上传目录
$targetDir = '../public/static/course/other/'; // 服务器初始缓存目录
$uploadDir = '../public/static/course/other/'; // 服务器初始上传目录
$servicePath = '/course/other/'; // 又拍云初始文件地址
if($path){
if(in_array($exe,$videoExe)){
$targetDir = '../public/static/course/'.$path.'/video/'; // 服务器缓存目录
$uploadDir = '../public/static/course/'.$path.'/video/'; // 服务器上传目录
$servicePath = '/course/'.$path.'/video/'; // 又拍云文件地址
}
if(in_array($exe,$audioExe)){
$targetDir = '../public/static/course/'.$path.'/audio/'; // 服务器缓存目录
$uploadDir = '../public/static/course/'.$path.'/audio/'; // 服务器上传目录
$servicePath = '/course/'.$path.'/audio/'; // 又拍云文件地址
}
}
// 切片上传
$fileInfo = $UploadService->uploadBlock($targetDir,$uploadDir);
// 上传至又拍云
if(!empty($fileInfo['uploadPath'])){
$uploadPath = $fileInfo['uploadPath']; // 服务器文件路径
$resultPath = $UploadService->uploadYouPaiYun($uploadPath,$exe,$servicePath,true);
return json(['url'=>$UploadService->uPaiUrl,'path'=>$resultPath]);
}
}
控制器去调用上传方法，这里我将上传方法封装为了服务：
image.png
UploadService里面的代码为：
<?php
/**
* Created by PhpStorm.
* User: cdjyj21
* Date: 2018/6/2
* Time: 9:16
*/
namespace app\services;
use Upyun\Upyun;
use Upyun\Config;
class UploadService
{
// 视频类型数组
public $videoExe = ['3gp','mp4','rmvb','mov','avi','mkv'];
// 音频类型数组
public $audioExe = ['wma','mp3','midi'];
// 又拍云地址
public $uPaiUrl = 'http://xxxxxxxx.com';
/**
* 大文件切片上传
* @param string $targetDir
* @param string $uploadDir
* @return array
*/
public function uploadBlock($targetDir='',$uploadDir=''){
header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");
if ($_SERVER['REQUEST_METHOD'] == 'OPTIONS') {
exit; // finish preflight CORS requests here
}
if ( !empty($_REQUEST[ 'debug' ]) ) {
$random = rand(0, intval($_REQUEST[ 'debug' ]) );
if ( $random === 0 ) {
header("HTTP/1.0 500 Internal Server Error");
exit;
}
}
// header("HTTP/1.0 500 Internal Server Error");
// exit;
// 5 minutes execution time
@set_time_limit(5 * 60);
// Uncomment this one to fake upload time
// usleep(5000);
// Settings
// $targetDir = ini_get("upload_tmp_dir") . DIRECTORY_SEPARATOR . "plupload";
$cleanupTargetDir = true; // 开启文件缓存删除
$maxFileAge = 60*60*24; // 文件缓存时间超过时间自动删除
// 验证缓存目录是否存在不存在创建
if (!file_exists($targetDir)) {
@mkdir($targetDir,0777,true);
}
// 验证缓存目录是否存在不存在创建
if (!file_exists($uploadDir)) {
@mkdir($uploadDir,0777,true);
}
// Get 或 file 方式获取文件名
if (isset($_REQUEST["name"])) {
$fileName = $_REQUEST["name"];
} elseif (!empty($_FILES)) {
$fileName = $_FILES["file"]["name"];
} else {
$fileName = uniqid("file_");
}
$oldName = $fileName;//记录文件原始名字
$filePath = $targetDir . $fileName;
// $uploadPath = $uploadDir . DIRECTORY_SEPARATOR . $fileName;
// Chunking might be enabled
$chunk = isset($_REQUEST["chunk"]) ? intval($_REQUEST["chunk"]) : 0;
$chunks = isset($_REQUEST["chunks"]) ? intval($_REQUEST["chunks"]) : 1;
// 删除缓存校验
if ($cleanupTargetDir) {
if (!is_dir($targetDir) || !$dir = opendir($targetDir)) {
die('{"jsonrpc" : "2.0", "error" : {"code": 100, "message": "Failed to open temp directory."}, "id" : "id"}');
}
while (($file = readdir($dir)) !== false) {
$tmpfilePath = $targetDir . $file;
// If temp file is current file proceed to the next
if ($tmpfilePath == "{$filePath}_{$chunk}.part" || $tmpfilePath == "{$filePath}_{$chunk}.parttmp") {
continue;
}
// Remove temp file if it is older than the max age and is not the current file
if (preg_match('/\.(part|parttmp|mp4)$/', $file) && (@filemtime($tmpfilePath) < time() - $maxFileAge)) {
@unlink($tmpfilePath);
}
}
closedir($dir);
}
// 打开并写入缓存文件
if (!$out = @fopen("{$filePath}_{$chunk}.parttmp", "wb")) {
die('{"jsonrpc" : "2.0", "error" : {"code": 102, "message": "Failed to open output stream."}, "id" : "id"}');
}
if (!empty($_FILES)) {
if ($_FILES["file"]["error"] || !is_uploaded_file($_FILES["file"]["tmp_name"])) { // 指定的文件是否是通过 HTTP POST 上传的
die('{"jsonrpc" : "2.0", "error" : {"code": 103, "message": "Failed to move uploaded file."}, "id" : "id"}');
}
// Read binary input stream and append it to temp file
if (!$in = @fopen($_FILES["file"]["tmp_name"], "rb")) {
die('{"jsonrpc" : "2.0", "error" : {"code": 101, "message": "Failed to open input stream."}, "id" : "id"}');
}
} else {
if (!$in = @fopen("php://input", "rb")) {
die('{"jsonrpc" : "2.0", "error" : {"code": 101, "message": "Failed to open input stream."}, "id" : "id"}');
}
}
while ($buff = fread($in, 50 * 1024 * 1024)) {
fwrite($out, $buff);
}
@fclose($out);
@fclose($in);
rename("{$filePath}_{$chunk}.parttmp", "{$filePath}_{$chunk}.part");
$index = 0;
$done = true;
for( $index = 0; $index < $chunks; $index++ ) {
if ( !file_exists("{$filePath}_{$index}.part") ) {
$done = false;
break;
}
}
//文件全部上传 执行合并文件
if ( $done ) {
$pathInfo = pathinfo($fileName);
$hashStr = substr(md5($pathInfo['basename']),8,16);
$hashName = time() . $hashStr . '.' .$pathInfo['extension'];
$uploadPath = $uploadDir .$hashName;
if (!$out = @fopen($uploadPath, "wb")) {
die('{"jsonrpc" : "2.0", "error" : {"code": 102, "message": "Failed to open output stream."}, "id" : "id"}');
}
if ( flock($out, LOCK_EX) ) {
for( $index = 0; $index < $chunks; $index++ ) {
if (!$in = @fopen("{$filePath}_{$index}.part", "rb")) {
break;
}
while ($buff = fread($in, 50 * 1024 * 1024)) {
fwrite($out, $buff);
}
@fclose($in);
@unlink("{$filePath}_{$index}.part");
}
flock($out, LOCK_UN);
}
@fclose($out);
return ['uploadPath'=>$uploadPath]; // 抛出文件路径
}
}
/**
* 上传又拍云
* @param $uploadPath
* @param $exe
* @param $servicePath
* @param $unlink
* @return string
*/
public function uploadYouPaiYun($uploadPath,$exe,$servicePath,$unlink){
// 上传又拍云
$bucketConfig = new Config('xxx', 'xxx', 'xxx'); // 配置参数，详情请见又拍云官方文档
$client = new Upyun($bucketConfig); // 实例化又拍云 传入配置参数
$file = fopen($uploadPath, 'r');
$name = md5(time().randomChars(10));
$serviceFile = $servicePath.$name.'.'.$exe;
$client->write($serviceFile, $file);
// 是否删除本地服务器上的视频
if($unlink){
@unlink($uploadPath);
}
return $serviceFile;
}
/**
* 删除又拍云上的文件
* @param $path
*/
public function delFileUPaiYun($path){
$bucketConfig = new Config('xxx', 'xxx', 'xxx'); // 配置参数，详情请见又拍云官方文档
$client = new Upyun($bucketConfig); // 实例化又拍云 传入配置参数
$client->delete($path);
}
}
即：通过 uploadBlock 方法将大文件切片上传至服务器，然后将每一个小文件重新整合。然后在通过 uploadYouPaiYun 方法将大文件上传至又拍云。
3、文件断续上传至又拍云浅析
先composer引入又拍云的包：
image.png
我们先来看看又拍云的
Upyun.php 文件里面的 write 方法
public function write($path, $content, $params = array(), $withAsyncProcess = false)
{
if (!$content) {
throw new \Exception('write content can not be empty.');
}
$upload = new Uploader($this->config);
$response = $upload->upload($path, $content, $params, $withAsyncProcess);
if ($withAsyncProcess) {
return $response;
}
return Util::getHeaderParams($response->getHeaders());
}
可见其调用了 Uploader.php 文件里面的 upload 方法
public function upload($path, $file, $params, $withAsyncProcess)
{
$stream = Psr7\stream_for($file);
$size = $stream->getSize();
$useBlock = $this->needUseBlock($size);
if ($withAsyncProcess) {
$req = new Form($this->config);
return $req->upload($path, $stream, $params);
}
if (! $useBlock) {
$req = new Rest($this->config);
return $req->request('PUT', $path)
->withHeaders($params)
->withFile($stream)
->send();
} else {
return $this->pointUpload($path, $stream, $params);
}
}
upload 方法调用了 needUseBlock 方法来进行是否需要断续上传的判断。
private function needUseBlock($fileSize)
{
if ($this->config->uploadType === 'BLOCK') {
return true;
} elseif ($this->config->uploadType === 'AUTO' &&
$fileSize >= $this->config->sizeBoundary) {
return true;
} else {
return false;
}
}
如果判断文件需要断续上传的话，则调用Uploader.php 文件里面的 pointUpload 方法
/**
* 断点续传
* @param $path
* @param $stream
* @param $params
*
* @return mixed|\Psr\Http\Message\ResponseInterface
* @throws \Exception
*/
private function pointUpload($path, $stream, $params)
{
$req = new Rest($this->config);
$headers = array();
if (is_array($params)) {
foreach ($params as $key => $val) {
$headers['X-Upyun-Meta-' . $key] = $val;
}
}
$res = $req->request('PUT', $path)
->withHeaders(array_merge(array(
'X-Upyun-Multi-Stage' => 'initiate',
'X-Upyun-Multi-Type' => Psr7\mimetype_from_filename($path),
'X-Upyun-Multi-Length' => $stream->getSize(),
), $headers))
->send();
if ($res->getStatusCode() !== 204) {
throw new \Exception('init request failed when poinit upload!');
}
$init = Util::getHeaderParams($res->getHeaders());
$uuid = $init['x-upyun-multi-uuid'];
$blockSize = 1024 * 1024;
$partId = 0;
do {
$fileBlock = $stream->read($blockSize);
$res = $req->request('PUT', $path)
->withHeaders(array(
'X-Upyun-Multi-Stage' => 'upload',
'X-Upyun-Multi-Uuid' => $uuid,
'X-Upyun-Part-Id' => $partId
))
->withFile(Psr7\stream_for($fileBlock))
->send();
if ($res->getStatusCode() !== 204) {
throw new \Exception('upload request failed when poinit upload!');
}
$data = Util::getHeaderParams($res->getHeaders());
$partId = $data['x-upyun-next-part-id'];
} while ($partId != -1);
$res = $req->request('PUT', $path)
->withHeaders(array(
'X-Upyun-Multi-Uuid' => $uuid,
'X-Upyun-Multi-Stage' => 'complete'
))
->send();
if ($res->getStatusCode() != 204 && $res->getStatusCode() != 201) {
throw new \Exception('end request failed when poinit upload!');
}
return $res;
}
网友评论